Subscribe to the feed
Linux 

Hopefully, you've read my 10 Basic Linux commands you need to know article, and now you're ready for the next higher rung on the sysadmin ladder. n this article, I explore commands that every system administrator should know for troubleshooting, general housekeeping, and daily activities that you must perform.

When you practice commands that can be harmful to a production system, have a virtual machine running somewhere that you can torture and restore should something go wrong. For some reason, people generally frown on having to repair or reinstall production systems because someone practiced a new command that went awry. Plus, it's cool to show up one day armed with new sysadmin skills to impress (school) your coworkers. Remember to say, "Watch this," to be sure they're paying attention before you hit the Enter key so it's more dramatic and awe-inspiring.

NOTE: You don't have to be the root user to run any of these commands. To change system parameters or to edit system files, though, you will have to be root.

Show who is logged in

As a system administrator, it's your job to keep track of who logs into your systems, either through automation or when you're in the system yourself. A quick check can tell you a lot about what's going on at that point. For example, if you have a system whose performance is "in the red" and you're not sure why issue the who command to find out who is logged in. If you see a developer or group of developers, they might be testing a new application that is grabbing all the resources. Or you might have the occasional rogue user running a poorly constructed Nmap command.

The who command tells you who is logged in, when they logged in, where they're logged in from, and even which type of connection they're using:

$ who

root tty1 2019-07-23 07:58
khess pts/0 2019-07-23 07:59 (192.168.1.81)

The ttyX logins are from the console and the pts/X ones are over the network from a computer via SSH. An acronym for Pseudo Terminal Slave, most sysadmins refer to the pts entries as pseudoterminals. The important thing is to note the difference between TTY (local console) and PTS (remote SSH) logins.

Another reason to run who is if you're about to perform system maintenance. A quick check will tell you who you have to contact to advise them to log out of the system because your maintenance might include a reboot or other activity that will disrupt their work.

echo a line of text

Believe it or not, echo is one of the most powerful commands at your disposal. With this command, you can do things like create files, append to them, check return codes, and view system variables.

To create a new file this command, use echo with some text, and then redirect the output to the file you want to create:

$ echo "This is a test file" > test.txt

You don't have to use quotes around the text, but I always do—I worry that the text I redirect to the file won't look right if I don't. To be sure it's correct, cat the file:

$ cat test.txt

This is a test file

To append some text on the next line, use the append redirect operator ( >> ):

$ echo "This is how to add text to a file" >> test.txt

$ cat test.txt

This is a test file
This is how to add text to a file

Check the return code from the last command you ran with echo:

$ echo $? 0

response typically means success. You can also use echo to check your environment variables:

$ echo $SHELL

/bin/bash

The echo man page gives you many more options and capabilities, such as how to use tabs, backspace, carriage returns, and more.

Display the top Linux processes

The top command does much more than simply display Linux processes, but it's a start. Run top at the command line to observe for yourself all the information that this command provides: 

top - 10:14:04 up 5 days, 48 min,  2 users,  load average: 0.00, 0.00, 0.02
Tasks: 233 total,   1 running, 232 sleeping,   0 stopped,   0 zombie
%Cpu(s):  5.9 us,  5.9 sy,  0.0 ni, 88.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   1829.4 total,    191.2 free,   1066.0 used,    572.2 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.    538.7 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                     
 1651 khess     20   0   64016   4936   4056 R  11.8   0.3   0:00.02 top                                                         
    1 root      20   0  179492  12076   6804 S   0.0   0.6   0:40.77 systemd                                                     
    2 root      20   0       0      0      0 S   0.0   0.0   0:00.17 kthreadd                                                    
    3 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_gp                                                      
    4 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_par_gp       

The listing above shows the first few lines from my Red Hat 8.0 server's top display. This command runs continuously, so to exit, type q. This command is named top in the first place because it provides a real-time display of the top processes by CPU and memory usage. To see an exhaustive explanation of the top command, refer to the man page.

Other than q, the most beneficial key command for me is k, which prompts a process ID (PID) to kill (terminate). As a system administrator, it is your job to protect system health for the general user population of that system. In other words, killing processes to release resources is one of the things you do, within reason, of course. It's career-limiting to kill processes in a haphazard fashion, but it's something that has to be done from time to time—killing processes—not killing them haphazardly.

The top command gives you a real-time snapshot of system performance. Typically, you run top when a performance problem is reported. When a system is idle, running top isn't exciting and often results in showing top as the most resource-consuming process on the system. Don't be alarmed by this, but do realize that it's possible.

Use top as much as you like, but realize that its information is not necessarily indicative of overall system performance. It is a snapshot and not a measure of long-term activity.

kill a process

Although I wrote in the section above that it's your job to sometimes kill processes, exercise caution when doing so. There's a good chance that abruptly ending a process will cause data corruption, data loss, and even job loss for you if you haven't cleared such actions through the proper channels.

The two most often used signals or options for the kill command are -15 and -9. Issuing a kill -15 <PID> is known as a soft, or polite kill. The -15 (also known as SIGTERM) signal kills the process but allows it to finish any pending processing:

$ kill -15 <PID>

The -9 signal ( SIGKILL ) immediately terminates the program with no regard for current processing. The -9 signal kills it. End of story. End of process:

$ kill -9 <PID>

There are two specific times to use the -9 signal. The first is when you have a runaway process that can't be killed with the -15 signal, and the second is when you need to free system resources immediately without regard for data loss or corruption. This second scenario is rare, but it does happen. In that situation, the only other option might be to reboot the system. Even after killing the process, you might have to reboot anyway—killing certain processes can leave the system in an unstable state.

The takeaway here is to use kill sparingly and only with permission.

Closely associated with the kill command is the killall command. If you have a process such as the Chrome web browser that can consume more than its share of resources, you can issue the killall <processname> command to rid the system of all its spawned processes. The killall command doesn't require you to know the PID, nor do you have to kill each individual process. Doing so can become way too tedious, and system administrators haven't the patience for such things.

$ killall chrome

This command terminates all instances of Chrome owned by this user. You can issue the same command as root, but read the previous dialog about exercising caution when doing so, because issuing such a command as root terminates the program for everyone on the system.

Note: If your system doesn't have the killall command available, then you'll have to add it by installing the psmisc package as shown below.

$ sudo yum -y install psmisc

I know, we haven't discussed the yum or dnf commands yet. Take this one as a "just do it" lesson at this point.

View files more or less

If you've used commands such as ps, you know that file listings can be long, and a lot of the information flows right off the screen. Sure, you can page up or scroll, but it's not very efficient.

The commands more and less limit the amount of data you see to one "page." As with many things Linux-related, users are in two camps: the more camp and the less camp. I'm in the more camp. I never use less. And, no, less isn't more. Even the less man page reads, "The opposite of more."

From a usage standpoint, these two commands are similar. However, the differences surface when interacting with these commands. It's impossible to show effectively in a static article but less has a few more navigation options than more. The more command's options are:

  • Advance one line using the Enter key.
  • Advance a full page using the Spacebar .
  • Quit by entering q.

You cannot move backward using moreLess , being more Wonkavator-esque , allows you to move backward, search for strings, and much more. Use the man pages for more and less to decide which of these commands is right for you.

To make things even more complex, there are two ways to use more and less. You can pipe (|) output to more and less or you can use these commands to operate directly on files. Here are some examples (without their output):

$ more /etc/passwd

$ cat /etc/passwd | more

$ ps -ef | more

$ less /etc/passwd

$ cat /etc/passwd | less

$ ps -ef | less

Update user passwd authentication tokens

Standard users use the passwd command to change their passwords. It's quick and simple to do. Issue the passwd command and you're prompted to change your password:

$ passwd

Changing password for user khess.
Current password:
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

When changing your password, you'll notice that the system does not respond with any dots, stars, or even blank spaces. This feature is far more secure in situations where someone is shoulder surfing during a password change. There is also no option for showing the password. Again, very secure.

There are additional passwd command options for the root user. For example, if you issue the following command as yourself, check your system's response:

$ passwd -S

Only root can do that.

If the root user issues this command with a username, the command displays user information: 

$ sudo passwd -S khess

khess PS 2019-07-29 0 99999 7 -1 (Password set, SHA512 crypt.)

The real power for system administrators is being able to set a user's password without knowing the current one:

$ sudo passwd khess

New password:
Retype new password:
passwd: all authentication tokens updated successfully.

As root, you can optionally lock and unlock user accounts:

$ sudo passwd -l john

Locking password for user john.
passwd: Success

$ sudo passwd -u john

Unlocking password for user john.
passwd: Success

Use passwd responsibly. And when offboarding a user, you should lock the account rather than deleting it: The user might have important data saved in their home directory, or have a process running that requires the account to be functional. Locking is good enough to prevent further interactive logins, and will also inform you about any automated tasks that require a password to perform.

ifconfig a network interface

There are tasks that as a sysadmin you don't do every day, but when you do them you need a power command like ifconfig. I classify this command in the power category because it does many things, but with simple syntax. 

Note: While a user can look at network interface configurations and settings with, you must be root to make changes.

$ ifconfig
enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.96  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 2600:1702:a40:88b0:581f:ea48:4e1a:6711  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::3d1d:ee56:9c1c:33b  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:a7:47:25  txqueuelen 1000  (Ethernet)
        RX packets 1153803  bytes 230635486 (219.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 78485  bytes 8389458 (8.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 48  bytes 5616 (5.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 48  bytes 5616 (5.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:7a:a9:b2  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

You can also use ifconfig to assign IP addresses to interfaces, change an interface's IP addresses, take an interface offline, bring one online, and more.

grep a pattern

Use the grep utility to search for a particular pattern inside a file or group of files. For example, say that you have a file in your home directory that contains the IP address of a remote system that you worked on a few months ago, but you can't recall the exact address. You know it was something like 192.168.10.???. The problem is that you have 50 files in your home directory and it would take hours to search through them all by hand.

Well, fret no more, grep is here to help. In this example, you can grep for the 192.168.10.pattern in your home directory:

$ grep 192.168.10. *

grep: data: Is a directory
grep: docs: Is a directory
grep: documents: Is a directory
grep: form: Is a directory
grep: forms: Is a directory

Notice that several of the entries state that you're attempting to look into files that are directories, and that your search came up negative for a file containing the IP address. Use the recursive option ( -R ) to search subdirectories:

$ grep -R 192.168.10. *

documents/systems_list.txt: 192.168.10.45 pumba

Here, your search was successful. The grep command returned the entire line that matches your pattern.

When system administrators mention grep or "grepping" something, they usually refer to piping in the same sentence, as in "Pipe it to grep." You don't always need to pipe to grep as you can see from the example above. But, piping to grep works in a similar way. To search for systemd in your process list:

$ ps -ef |grep systemd
root         1     0  0 Aug07 ?        00:00:40 /usr/lib/systemd/systemd --switched-root --system --deserialize 17
root       476     1  0 Aug07 ?        00:00:11 /usr/lib/systemd/systemd-journald
root       505     1  0 Aug07 ?        00:00:02 /usr/lib/systemd/systemd-udevd
root       632     1  0 Aug07 ?        00:00:02 /usr/lib/systemd/systemd-machined
dbus       653     1  0 Aug07 ?        00:01:45 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root       712     1  0 Aug07 ?        00:00:07 /usr/lib/systemd/systemd-logind
gdm       1209     1  0 Aug07 ?        00:00:02 /usr/lib/systemd/systemd --user
gdm       1301  1209  0 Aug07 ?        00:00:00 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
khess     2423 29513  0 10:25 pts/1    00:00:00 grep --color=auto systemd
khess     8088     1  0 Aug07 ?        00:00:03 /usr/lib/systemd/systemd --user
khess     8113  8088  0 Aug07 ?        00:00:00 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only

As you can see, piping to grep is the only way you can find all instances of systemd from the process list. Note the first entry with my username on it. That is my grep command searching for systemd. If you don't want to see that entry, use the -v option to exclude the grep command itself from your results:

$ ps -ef | grep systemd | grep -v grep

The other grep option that I find helpful is the ignore case option( -i ):

$ grep -iR bob *

This command searches recursively through all files for the string bob , regardless of case, which could match all of the following: Bob Spongebobbilbobaggins, and BObrice.

Grep is very useful and can be used on text files, and in conjunction with other commands via piping. You can also grep for complex patterns using regular expressions (regex) but that is a topic for other articles.

Scan and process patterns with awk

I feel like awk is one of those tools that few people use because they don't understand the full power and possibilities of this little dynamo. I will jump right in with some examples. Say that I want a list of all processes that are systemd-related, but I only want the PIDs, not all of the other information that you get with ps:

$ ps -ef | grep systemd | grep -v grep | awk '{print $2}'

1
471
608
631
7449
7494
32681

To explain the command above: I ran a ps, grepped for systemd, removed my own grep command, and then piped the output to awk and printed the second column. It is the second column because by default awk uses a space as a field separator. The formal awk part of the command would look like this: awk -F " " '{print $2}', where the -F option defines the field separator. For comma-separated values, you'd use: awk -F "," '{print $2}'.

If you have a text file ( test.txt) containing the following:

one,two,three,four,five
1,2,3,4,5
6,7,8,9,10
a,b,c,d,e

And you run awk against that file to extract the third column of data, it displays the following:

$ cat test.txt | awk -F "," '{print $3}'

three
3
8
c

I think you can see what's going on with awk here. It's handy for automation scripting as you can probably tell from these examples. You can extract data and operate on it dynamically with awk.

Edit text with vi

The vi (visual) text editor was a clever developer's (Bill Joy) answer to updating the old line editor ex, which Bill Joy also wrote. This program 40+ years later is still the most used Linux command line text editor.

The vi editor is small, with the latest incarnation ( vim aka vi improved) weighing at just over 3MB in size. These days vi is often a symbolic link to vim (in RHEL 8, for example). Its enhancements include multi-level undo, multiple windows and buffers, syntax highlighting, command line editing, file name completion, online help, and visual selection. Open vim and use the following command for a summary of the differences between vim and vi 

:help vi_diff.txt

vi has so many options and features that I'm only mentioning it as one of the commands you need to know in this article. Please refer to my vi : An introduction article for a more extensive look at vi.

Wrapping up

Surprisingly, out of more than 200 possible Linux commands, most system administrators only use about two dozen on a regular basis. If you know those, system administration becomes easier and far more elegant. Struggling with commands and syntax makes the job harder. Learn these popular and highly-used commands and you'll have the power to make a difference in your environment.

[Want to try out Red Hat Enterprise Linux? Download it now for free.]


About the author

Ken has used Red Hat Linux since 1996 and has written ebooks, whitepapers, actual books, thousands of exam review questions, and hundreds of articles on open source and other topics. Ken also has 20+ years of experience as an enterprise sysadmin with Unix, Linux, Windows, and Virtualization.

Follow him on Twitter: @kenhess for a continuous feed of Sysadmin topics, film, and random rants.

In the evening after Ken replaces his red hat with his foil hat, he writes and makes films with varying degrees of success and acceptance. He is an award-winning filmmaker who constantly tries to convince everyone of his Renaissance Man status, also with varying degrees of success and acceptance.

Read full bio
UI_Icon-Red_Hat-Close-A-Black-RGB

Browse by channel

automation icon

Automation

The latest on IT automation for tech, teams, and environments

AI icon

Artificial intelligence

Updates on the platforms that free customers to run AI workloads anywhere

open hybrid cloud icon

Open hybrid cloud

Explore how we build a more flexible future with hybrid cloud

security icon

Security

The latest on how we reduce risks across environments and technologies

edge icon

Edge computing

Updates on the platforms that simplify operations at the edge

Infrastructure icon

Infrastructure

The latest on the world’s leading enterprise Linux platform

application development icon

Applications

Inside our solutions to the toughest application challenges

Original series icon

Original shows

Entertaining stories from the makers and leaders in enterprise tech