5 Linux network troubleshooting commands
Networking configuration and troubleshooting are crucial tasks that sysadmins need to perform regularly. Some of these tasks can be challenging. However, when dealing with connectivity issues, using the right tools will assist you in achieving the results in a faster and consistent way.
The ip command
ip command is an all-around utility to show and manipulate network objects on your Linux system, including IP addresses, routes, and ARP tables. It's a useful tool to configure the network, as well as to troubleshoot network connectivity issues.
ip command replaces the functionality of many commands provided with the old
net-tools package such as
arp, but it adds many other features.
In its most basic form, you can just run
ip and provide a network object to manipulate, such as an address, link, or route, and a subcommand to perform an action. If you do not give a subcommand, many objects default to the
show subcommand to display information related to that object.
Here is the basic syntax:
ip <OBJECT> [COMMAND]
For example, to see the link status on all network devices, run
ip link show:
$ ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether 52:54:00:b5:c7:2b brd ff:ff:ff:ff:ff:ff
show is the default subcommand, you can also obtain the same result by running
ip link or even
ip l (many objects recognize an abbreviation).
ip command manages many objects. The main ones are:
l- controls the status of network devices
a- manipulates IP addresses on devices
r- handles routing table entries
n- controls ARP table entries
You can see a full list of objects and commands by running
Some useful examples of the ip command
Show network statistics
-s in human readable format
-h for a specific network interface:
$ ip -s -h l show dev enp1s0 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether 52:54:00:b5:c7:2b brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 820M 303k 0 182k 0 0 TX: bytes packets errors dropped carrier collsns 19.9M 60.9k 0 0 0 0
Show the IP addresses of all interfaces:
$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:b5:c7:2b brd ff:ff:ff:ff:ff:ff inet 192.168.122.169/24 brd 192.168.122.255 scope global noprefixroute enp1s0 valid_lft forever preferred_lft forever inet6 fe80::7ecf:5cc8:5c1f:1009/64 scope link noprefixroute valid_lft forever preferred_lft forever
Show the IP address of a single interface e.g., enp1s0:
$ ip a show dev enp1s0
Add another IP address to an interface (requires
$ sudo ip a change 192.168.122.170 dev enp1s0
Display the routing table:
$ ip route default via 192.168.122.1 dev enp1s0 proto static metric 100 192.168.122.0/24 dev enp1s0 proto kernel scope link src 192.168.122.169 metric 100
Add a route (default gateway):
$ sudo ip route add default via 192.168.122.1 dev enp1s0
ip command is a lower-level interface to configure network options on your Linux system. While it's useful as a troubleshooting tool, it may be harder to set the network with it. For this reason, many distributions provide a higher-level interface to accomplish the same task. Next, let's take a look at the Network Manager configuration tool
nmcli, which provides an integrated way to configure the network.
The nmcli tool
Network Manager is a network configuration application available by default with many Linux distributions, including RHEL and Fedora. Network Manager runs as a daemon, and its goal is to provide a higher-level interface to make network configuration easier and more automated.
It's common for users of desktop Linux or servers with a graphical interface to use Network Manager GUI clients to configure the network. For cases where you work with headless servers or if you want to automate configuration via shell scripts, the
nmcli tool comes in handy.
Network Manager and
nmcli are a versatile combination that allow you to verify and configure many network options. The basic syntax for
nmcli <OBJECT> [COMMAND] [ARGUMENTS]
The most common objects are:
general- shows Network Manager status and permissions
networking- shows, enables, and disables networking
radio- shows, enables, and disables WiFi and WWAN
device- shows and manipulates the status of network devices
connection- manages Network Manager connection profiles
It's impossible to cover all the options on a single blog post. Let's check some examples of how to use
nmcli for common network configurations tasks. For more examples, consult the official nmcli-examples page or the man pages with
Check the status of network devices:
$ nmcli device status DEVICE TYPE STATE CONNECTION enp1s0 ethernet connected enp1s0 enp7s0 ethernet disconnected -- lo loopback unmanaged --
Show all connection profiles:
$ nmcli con show NAME UUID TYPE DEVICE enp1s0 1bb35a4a-ad02-4cad-978a-4a97ea9527cb ethernet enp1s0
Show details about a specific connection (e.g., enp1s0):
$ nmcli connection show enp1s0
Retrieve specific parameters from a connection (e.g., IP address and DNS):
$ nmcli -g ip4.address,ip4.dns connection show enp1s0 192.168.122.169/24 192.168.122.1
Modify connection parameters, e.g., switch from DHCP to manual connection:
$ sudo nmcli con mod enp1s0 ipv4.method manual ipv4.addresses 192.168.122.169/24 ipv4.dns 192.168.122.1 ipv4.gateway 192.168.122.1
Restart the connection to enable new options:
$ sudo nmcli con down enp1s0 $ sudo nmcli con up enp1s0
Add a new connection profile:
$ sudo nmcli connection add con-name enp7s0 ifname enp7s0 type ethernet ip4 192.168.64.88/24
nmcli also allows you to edit a connection profile interactively. This interface provides help and auto-completion via the Tab key, which guides you through the many options available. Activate the editor using
nmcli connection edit CONNECTION-NAME:
$ sudo nmcli connection edit enp7s0 ===| nmcli interactive connection editor |=== Editing existing '802-3-ethernet' connection: 'enp7s0' Type 'help' or '?' for available commands. Type 'print' to show all the connection properties. Type 'describe [<setting>.<prop>]' for detailed property description. You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, dcb, sriov, ethtool, match, ipv4, ipv6, tc, proxy.
In the editor, you can define specific parameters with the
set command. Use the Tab key auto-completion to see all available options:
nmcli> set ipv4.<PRESS TAB> addresses dhcp-hostname dhcp-timeout dns-search may-fail routes dad-timeout dhcp-hostname-flags dns gateway method route-table dhcp-client-id dhcp-iaid dns-options ignore-auto-dns never-default routing-rules dhcp-fqdn dhcp-send-hostname dns-priority ignore-auto-routes route-metric nmcli> set ipv4.addresses 192.168.64.90/24
Then you can
nmcli> print ipv4.addresses ipv4.addresses: 192.168.64.88/24, 192.168.64.90/24
When you finish your changes,
save them to the connection and
quit to complete the operation:
nmcli> save Connection 'enp7s0' (94170029-5620-4f90-ad78-704b21480b1a) successfully updated. nmcli> quit
Finally, restart the connection to make the changes effective.
Network Manager and
nmcli are a complete solution for network configuration. If you have never used it before, start with some of the basic commands and gradually move to more complex settings. Consult the man pages and examples to execute specific tasks.
Next, let's review some commands that help you with network troubleshooting, starting with name resolution.
The nslookup command
When managing connectivity issues, DNS name resolution is often a source of headaches. The
nslookup utility helps you check and troubleshoot DNS name resolution.
This command is available with the
bind-utils package on RHEL8 and Fedora systems. Install it with
$ sudo dnf install -y bind-utils
To quickly check the name resolution for a particular host, use
nslookup with the hostname as an argument. The command uses the default DNS configuration for the name resolution:
$ nslookup redhat.com Server: 192.168.122.1 Address: 192.168.122.1#53 Non-authoritative answer: Name: redhat.com Address: 18.104.22.168
If the name resolution fails, you can use an alternative name server for the resolution by providing its address as the third argument:
$ nslookup redhat.com 192.168.0.9 Server: 192.168.0.9 Address: 192.168.0.9#53 Non-authoritative answer: Name: redhat.com Address: 22.214.171.124
This information is useful for troubleshooting as it helps to identify whether the issue is local due to a specific server, or something more extensive.
You can also use it to run reverse DNS queries by providing the IP address instead of a hostname:
$ nslookup 126.96.36.199 188.8.131.52.in-addr.arpa name = redirect.redhat.com.
nslookup utility is a useful tool to help troubleshoot network issues related to DNS name resolution. Next, let's take a look at troubleshooting local network sockets.
The ss utility
Another common task when troubleshooting network connectivity issues is determining whether a connection is established or a particular service is available on a server.
ss command, short for socket statistics, is a convenient tool that displays network socket information. It's the modern replacement for
netstat that provides similar functionality, but includes a few extra features.
ss with no options to see the complete list of all established network sockets for socket types TCP, UDP, and UNIX.
$ ss Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port u_str ESTAB 0 0 * 25812 * 25811 u_str ESTAB 0 0 /run/systemd/journal/stdout 23604 * 23603 ... TRUNCATED OUTPUT u_str ESTAB 0 0 * 22566 * 22171 icmp6 UNCONN 0 0 *:ipv6-icmp *:* icmp6 UNCONN 0 0 *:ipv6-icmp *:* tcp ESTAB 0 0 192.168.122.169:ssh 192.168.122.1:45626
Since the default command displays all established network connections, this list can be extensive on a busy machine, with over a thousand entries. To help you with troubleshooting,
ss provides several filtering options.
You can display socket information for specific socket types with the following command-line parameters:
-t for TCP,
-u for UDP, and
-x for UNIX.
For example, display TCP socket information with
$ ss -t State Recv-Q Send-Q Local Address:Port Peer Address:Port ESTAB 0 0 192.168.122.169:ssh 192.168.122.1:45626
You can also filter the list by source or destination hostname or IP address. For example, here is a list of established connections for destination IP address 192.168.122.1:
$ ss dst 192.168.122.1 Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port tcp ESTAB 0 0 192.168.122.169:ssh 192.168.122.1:45626
These commands are useful to troubleshoot network connectivity in general or between specific hosts in the network. Another practical application for
ss is to verify whether network services are listening on the local machine with the correct address and port. To verify listening sockets, use the option
$ ss -l Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port nl UNCONN 0 0 rtnl:NetworkManager/909 * nl UNCONN 0 0 rtnl:systemd-resolve/1122 * ... TRUNCATED OUTPUT udp UNCONN 0 0 0.0.0.0:hostmon 0.0.0.0:* udp UNCONN 0 0 127.0.0.53%lo:domain 0.0.0.0: udp UNCONN 0 0 127.0.0.1:323 0.0.0.0:* udp UNCONN 0 0 [::]:hostmon [::]:* udp UNCONN 0 0 [::1]:323 [::]:* tcp LISTEN 0 128 0.0.0.0:ssh 0.0.0.0:* tcp LISTEN 0 128 0.0.0.0:hostmon 0.0.0.0:* tcp LISTEN 0 128 [::]:ssh [::]: tcp LISTEN 0 128 [::]:hostmon [::]:*
You can use the same filtering options described above to filter specific socket types. For example, to list all listening TCP sockets enter:
$ ss -lt State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 0.0.0.0:ssh 0.0.0.0:* LISTEN 0 128 0.0.0.0:hostmon 0.0.0.0: LISTEN 0 128 [::]:ssh [::]: LISTEN 0 128 [::]:hostmon [::]:
Instead of displaying the service name, like
ss can display port numbers by using the
$ ss -ltn State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 0.0.0.0:22 0.0.0.0: LISTEN 0 128 0.0.0.0:5355 0.0.0.0: LISTEN 0 128 [::]:22 [::]: LISTEN 0 128 [::]:5355 [::]:
Finally, another useful option is
-p to display process related information such as user name and process ID (PID). Some services may require elevated privileges via
sudo or as
root to list process information:
$ sudo ss -tnlp State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=919,fd=5)) LISTEN 0 128 0.0.0.0:5355 0.0.0.0:* users:(("systemd-resolve",pid=1122,fd=13)) LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=919,fd=7)) LISTEN 0 128 [::]:5355 [::]:* users:(("systemd-resolve",pid=1122,fd=15))
ss tool is an essential command in the sysadmin toolbox for network troubleshooting tasks. Next, let's take a look at
tracepath to trace network connectivity between hosts.
The tracepath command
tracepath command is a network troubleshooting tool that displays the network connectivity path between the local host and a remote host, identifying all routers used to route the traffic between them.
In case you're unable to connect to a network service in a remote host,
tracepath helps you determine where the issue is.
tracepath is a replacement for
traceroute, offering similar functionality. The main difference is that
tracepath uses random UDP ports instead of the ICMP protocol for the trace, thus not requiring elevated privileges to run.
For the basic usage, provide the hostname or IP address of the destination host. You can also provide the option
-n to display IP addresses for the routers instead of their names:
$ tracepath -n sat65server 1?: [LOCALHOST] pmtu 1500 1: 192.168.122.1 0.415ms 1: 192.168.122.1 0.299ms 2: 192.168.10.10 0.904ms 3: 192.168.88.1 1.127ms 4: 192.168.0.95 2.020ms Resume: pmtu 1500
tracepath cannot connect to a network hop, it displays no reply. By default, the maximum number of hops it tries is 30, which is usually enough. You can change that with the option
Suppose it receives no replies after a particular hop. That is a good indicator where to go next for your troubleshooting task.
tracepath information is not definitive, as the traffic could be blocked for several reasons. However, it helps you narrow down the issue and focus on the resolution efforts.
For security reasons, many routers on the Internet block traffic, so
tracepath may not be as useful for tracing connectivity with Internet services. It's still useful for local network troubleshooting.
In this article, we explored five essential network configuration and troubleshooting tools for Linux systems. These tools are powerful and provide many options that are hard to cover in a single post. We encourage you to check them out and look at documentation and man pages to see how you can incorporate them into your workflow.
There are other useful Linux network tools, such as
firewall-cmd. These tools are covered in other Enable Sysadmin articles.
[ Network hard to manage? Check out Network Automation for Everyone, a free book from Red Hat. ]