[libvirt] NL_RECV RETURNED WITH ERROR: NO BUFFER SPACE AVAILABLE ERROR FOR IXGBE LOAD
vaughan
vaughan.cao at oracle.com
Tue Jun 17 03:16:02 UTC 2014
Hi experts,
Release: OL7
Kernel: 3.10.0-121.el7.x86_64
Noticed below error on OL7 server, while loading Intel 10gigabit nic
driver module , ixgbe in syslog
--------------------------------------------------------------------
journal: nl_recv returned with error: No buffer space available
-------------------------------------------------------------------
Complete syslog content for ixgbe module load :
un 16 20:46:10 ca-ostest432 kernel: ixgbe: Intel(R) 10 Gigabit PCI
Express Network Driver - version 3.15.1-k
Jun 16 20:46:10 ca-ostest432 kernel: ixgbe: Copyright (c) 1999-2013
Intel Corporation.
Jun 16 20:46:10 ca-ostest432 kvm: 1 guest now active
Jun 16 20:46:10 ca-ostest432 kvm: 0 guests now active
Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: Multiqueue
Enabled: Rx Queue count = 16, Tx Queue count = 16
Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: (PCI
Express:5.0GT/s:Width x8) 00:1b:21:c8:24:74
Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: MAC: 2, PHY: 9,
SFP+: 3, PBA No: E70856-007
Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: PCI Express
bandwidth of 32GT/s available
Jun 16 20:46:10 ca-ostest432 kernel: ixgbe 0000:13:00.0: (Speed:5.0GT/s,
Width: x8, Encoding Loss:20%)
Jun 16 20:46:10 ca-ostest432 journal: nl_recv returned with error: No
buffer space available
Jun 16 20:46:11 ca-ostest432 journal: nl_recv returned with error: No
buffer space available
Jun 16 20:46:11 ca-ostest432 systemd-udevd[4939]: renamed network
interface eth0 to ens2f0
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.0: Intel(R) 10
Gigabit Network Connection
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: Multiqueue
Enabled: Rx Queue count = 16, Tx Queue count = 16
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: (PCI
Express:5.0GT/s:Width x8) 00:1b:21:c8:24:75
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: MAC: 2, PHY: 9,
SFP+: 4, PBA No: E70856-007
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: PCI Express
bandwidth of 32GT/s available
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: (Speed:5.0GT/s,
Width: x8, Encoding Loss:20%)
Jun 16 20:46:11 ca-ostest432 systemd-udevd[4984]: renamed network
interface eth0 to ens2f1
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.1: Intel(R) 10
Gigabit Network Connection
Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0):
carrier is OFF (but ignored)
Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): new
Ethernet device (driver: 'ixgbe' ifindex: 14)
Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0):
exported as /org/freedesktop/NetworkManager/Devices/13
Jun 16 20:46:11 ca-ostest432 NetworkManager[1316]: <info> (ens2f0):
device state change: unmanaged -> unavailable (reason 'managed') [10 20 2]
Jun 16 20:46:11 ca-ostest432 journal: nl_recv returned with error: No
buffer space available
Jun 16 20:46:11 ca-ostest432 kernel: ixgbe 0000:13:00.0: registered PHC
device on ens2f0
Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_UP): ens2f0:
link is not ready
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f0):
preparing device
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1):
carrier is OFF (but ignored)
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): new
Ethernet device (driver: 'ixgbe' ifindex: 15)
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1):
exported as /org/freedesktop/NetworkManager/Devices/14
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1):
device state change: unmanaged -> unavailable (reason 'managed') [10 20 2]
Jun 16 20:46:12 ca-ostest432 journal: nl_recv returned with error: No
buffer space available
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1):
preparing device
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f0):
device state change: unavailable -> disconnected (reason 'none') [20 30 0]
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1):
device state change: unavailable -> disconnected (reason 'none') [20 30 0]
Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.0 ens2f0: detected
SFP+: 3
Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.1: registered PHC
device on ens2f1
Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_UP): ens2f1:
link is not ready
Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.1 ens2f1: detected
SFP+: 4
Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.1 ens2f1: NIC Link
is Up 10 Gbps, Flow Control: RX/TX
Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_CHANGE):
ens2f1: link becomes ready
Jun 16 20:46:12 ca-ostest432 NetworkManager[1316]: <info> (ens2f1): link
connected
Jun 16 20:46:12 ca-ostest432 kernel: ixgbe 0000:13:00.0 ens2f0: NIC Link
is Up 10 Gbps, Flow Control: RX/TX
Jun 16 20:46:12 ca-ostest432 kernel: IPv6: ADDRCONF(NETDEV_CHANGE):
ens2f0: link becomes ready
Jun 16 20:46:13 ca-ostest432 NetworkManager[1316]: <info> (ens2f0): link
connected
---------------------------------------------------------------------------------------------
Issue is not seen for other nic drivers when loaded , i.e verified with
be2net ,e1000e , igb drivers , so it looks like specific to ixgbe
driver module.
This problem can be addressed by increasing the size of the socket
receive buffer via nl_socket_set_buffer_size(). However, since reliable
transmissions from kernel to user are impossible in any case, I don't
know if we need to ensure enough buffer and how much it will be enough.
In my test, 32768*4 bytes of receive buffer size is ok. Below is my patch.
>From 4505e0001563bcef814dd2c6dc02b2d83e521867 Mon Sep 17 00:00:00 2001
From: rpm-build <rpm-build>
Date: Mon, 16 Jun 2014 20:05:08 +0800
Subject: [PATCH] set nl socket buffer size to 32768*4
Signed-off-by: Vaughan Cao <vaughan.cao at oracle.com>
---
src/util/virnetlink.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/src/util/virnetlink.c b/src/util/virnetlink.c
index 5849ed7..cf44ad2 100644
--- a/src/util/virnetlink.c
+++ b/src/util/virnetlink.c
@@ -561,6 +561,15 @@ virNetlinkEventServiceStart(unsigned int protocol,
unsigned int groups)
goto error_server;
}
+ if (nl_socket_set_buffer_size(srv->netlinknh, (32768*4), 0)) {
+ virReportSystemError(errno, "%s",
+ _("cannot set netlink socket buffer size to 32768*4"));
+ goto error_server;
+ } else {
+ virReportSystemError(0, "%s",
+ _("set netlink socket buffer size to 32768*4"));
+ }
+
if ((srv->eventwatch = virEventAddHandle(fd,
VIR_EVENT_HANDLE_READABLE,
virNetlinkEventCallback,
--
1.9.0
--
Regards,
Vaughan
More information about the libvir-list
mailing list