TCP can have a large effect on latency. TCP adds latency in order to obtain efficiency, control congestion, and to ensure reliable delivery. When tuning, consider the following points:
Do you need ordered delivery?
Do you need to guard against packet loss?
Packet loss is not always bad. Transmitting the packet again can cause greater delays.
If you must use TCP, consider disabling the Nagle buffering algorithm by using TCP_NODELAY on your socket. The Nagle algorithm collects small outgoing packets to send all at once, and can have a detrimental effect on latency.
There are numerous tools for tuning the network. Here are some of the more useful:
To reduce netwrok traffic, packets can be collected and a single interrupt generated.
Use the -C (--coalesce) option with the ethtool command to enable.
Often, I/O switches can be subject to back-pressure, where network data builds up as a result of full buffers.
Use the -A (--pause) option with the ethtool command to change pause parameters and avoid network congestion.
Infiniband is a type of communications architecture often used to increase bandwidth and provide quality of service and failover. It can also be used to improve latency through Remote Direct Memory Access (RDMA) capabilities.
Use the -s (--statistics) option with the netstat comamnd to monitor network traffic.
For more information, or for further reading, the following man pages are related to the information given in this section.
ethtool(8)
netstat(8)