Linux troubleshooting: Setting up a TCP listener with ncat

The life of a sysadmin is hectic, rushed, and often frustrating. So, what you really need is a toolbox filled with tools that you easily recognize and can use quickly without another learning curve when things are going bad. One such tool is the ncat command.

ncat - Concatenate and redirect sockets

The ncat command has many uses, but the one I use it for is troubleshooting network connectivity issues. It is a handy, quick, and easy to use tool that I can't live without. Follow along and see if you decide to add it to your toolbox as well.

From the ncat man page:

Ncat is a feature-packed networking utility which reads and writes data across networks from the command line. Ncat was written for the Nmap Project and is the culmination of the currently splintered family of Netcat incarnations. It is designed to be a reliable back-end tool to instantly provide network connectivity to other applications and users. Ncat will not only work with IPv4 and IPv6 but provides the user with a virtually limitless number of potential uses.

Among Ncat's vast number of features there is the ability to chain Ncats together; redirection of TCP, UDP, and SCTP ports to other sites; SSL support; and proxy connections via SOCKS4, SOCKS5 or HTTP proxies (with optional proxy authentication as well).

Firewall problem or something else?

You've just installed <insert network service here>, and you can't connect to it from another computer on the same network. It's frustrating. The service is enabled. The service is started. You think you've created the correct firewall exception for it, but yet, it doesn't respond.

Your troubleshooting life begins. In what can stretch from minutes to days to infinity and beyond, you attempt to troubleshoot the problem. It could be many things: an improperly configured (or unconfigured) firewall exception, a NIC binding problem, a software problem somewhere in the service's code, a service misconfiguration, some weird compatibility issue, or something else unrelated to the network or the service blocking access. This is your scenario. Where do you start when you've checked all of the obvious places?

The ncat command to the rescue

The ncat command should be part of your basic Linux distribution, but if it isn't, install the nmap-ncat package and you'll have the latest version of it. Check the ncat man page for usage, if you're interested in its many capabilities beyond this simple troubleshooting exercise.

Using the ncat command, you will set up a TCP listener, which is a TCP service that waits for a connection from a remote system on a specified port. The following command starts a listening socket on TCP port 9999.

$ sudo ncat -l 9999

This command will "hang" your terminal. You can place the command into background mode, to operate similar to a service daemon using the & (ampersand) signal. Your prompt will return.

$ sudo ncat -l 8080 &

From a remote system, use the following command to attempt a connection:

$ telnet <IP address of ncat system> 9999

The attempt should fail as shown:

Trying <IP address of ncat system>...
telnet: connect to address <IP address of ncat system>: No route to host

This might be similar to the message you receive when attempting to connect to your original service. The first thing to try is to add a firewall exception to the ncat system:

$ sudo firewall-cmd --add-port=9999/tcp

This command allows TCP requests on port 9999 to pass through to a listening daemon on port 9999.

Retry the connection to the ncat system:

$ telnet <IP address of ncat system> 9999

Trying <IP address of ncat system>...
Connected to <IP address of ncat system>.
Escape character is '^]'.

This message means that you are now connected to the listening port, 9999, on the remote system. To disconnect, use the keyboard combination, CTRL + ]. Type quit to return to a prompt.

$ telnet <IP address of ncat system> 9999

Trying <IP address of ncat system>...
Connected to <IP address of ncat system>.
Escape character is '^]'.
^]
telnet>quit
Connection closed.
$

Disconnecting will also kill the TCP listening port on the remote (ncat) system, so don't attempt another connection until you reissue the ncat command. If you want to keep the listening port open rather than letting it die each time you disconnect, issue the -k (keep open) option. This option keeps the listening port alive. Some sysadmins don't use this option because they might leave a listening port open potentially causing security problems or port conflicts with other services.

$ sudo ncat -k -l 9999 &

What ncat tells you

The success of connecting to the listening port of the ncat system means that you can bind a port to your system's NIC. You can successfully create a firewall exception. And you can successfully connect to that listening port from a remote system. Failures along the path will help narrow down where your problem is.

What ncat doesn't tell you

Unfortunately, there's no solution for connectivity issues in this troubleshooting technique that isn't related to binding, port listening, or firewall exceptions. This is a limited scope troubleshooting session, but it's quick, easy, and definitive. What I've found is that most connectivity issues boil down to one of these three. My next step in the process would be to remove and reinstall the service package. If that doesn't work, download a different version of the package and see if that works for you. Try going back at least two revisions until you find one that works. You can always update to the latest version after you have a working service.

Wrap up

The ncat command is a useful troubleshooting tool. This article only focused on one tiny aspect of the many uses for ncat. Troubleshooting is as much of an art as it is a science. You have to know which answers you have and which ones you don't have. You don't have to troubleshoot or test things that already work. Explore ncat's various uses and see if your connectivity issues go away faster than they did before.

[ Network getting out of control? Check out Network automation for everyone, a free book from Red Hat. ]