[libvirt] Error starting domain: internal error: missing IFLA_VF_INFO in netlink response

[Laine Stump]

On 01/08/2015 04:40 AM, Hong-Hua.Yin at freescale.com wrote:
> Hi Laine,
>
> Sorry to disturb you. 
> It seemed this issue had been fixed in libvirt-1.2.2/libnl-3.2.22/linux-3.12. But we still got the error on PowerPC platform.
> I'll appreciate if you could give any suggestion. We are not sure if any netlink implementation in kernel space is missed.
>
> The scenario is a little complicated. We installed internal PF and VF kernel modules and want to use 
> <interface type="hostdev" managed="yes"> syntax to start a guest domain with MAC address.
>
> # insmod fslinic.ko max_vfs=2
> Freescale 10 Gigabit PCI Express Network Driver
> fslinic 0000:01:00.0: Multiqueue Enabled: Rx Queue count = 1, Tx Queue count = 1
> fslinic 0000:01:00.0: Freescale (R) 10 Gigabit Network Connection
> fslinic 0000:01:00.1: Multiqueue Enabled: Rx Queue count = 1, Tx Queue count = 1
> fslinic 0000:01:00.1: Freescale (R) 10 Gigabit Network Connection
>
> # insmod fslinicvf.ko
> Freescale 10 Gigabit PCI Express Network Driver
>
> # lspci -mk
> 00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> 01:00.0 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> 01:00.1 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> 01:00.4 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" ""
> 01:00.5 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" ""
> 01:01.0 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" ""
> 01:01.1 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" ""
> 0001:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> 0002:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""

What is the output of "virsh nodedev-dumpxml pci_0000_01_00_4"? It
should contain an element like this:

    <capability type='phys_function'>
    ...

This indicates that it is an sriov VF (its PF will be listed in the
above <capability> element). If this element is missing, there is
something incomplete about the SRIOV implementation for your card's driver.


>
> # echo 1957 0000 > /sys/bus/pci/drivers/vfio-pci/new_id
> # echo 1957 1957 > /sys/bus/pci/drivers/vfio-pci/new_id


You shouldn't need to do the above when managed='yes', since libvirt
does it for you (doing it manually like this may or may not have adverse
consequences, I haven't tried it)


>
>     <interface type="hostdev" managed="yes">
>       <mac address="00:e0:0c:00:20:01"/>
>       <source>
>         <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x04"/>
>       </source>
>     </interface>
>     <memballoon model='virtio'/>
>   </devices>
> </domain>
>
> root at t4240rdb:/var/volatile# virsh start interface
> error: Failed to start domain interface
> error: internal error: missing IFLA_VF_INFO in netlink response

At some point (it's been quite awhile, so I've forgotten exactly when)
the kernel stopped automatically including the IFLA_VF_INFO object in
the response to a netlink RTM_GETLINK request. In order for this info to
be included, you need to add an IFLA_EXT_MASK with the RTEXT_FILTER_VF
bit set into the RTM_GETLINK request. This is done in
src/util/virnetdev.c:virNetDevLinkDump() -
here are two bugzilla records describing the problem and pointing at the
patches added to fix it:

 https://bugzilla.redhat.com/show_bug.cgi?id=889249
 https://bugzilla.redhat.com/show_bug.cgi?id=889319

That particular code isn't compiled into libvirt unless RTEXT_FILTER_VF
is #defined, which should be done in /usr/include/linux/rtnetlink.h. If
the kernel headers used when building libvirt were missing that #define
(or maybe instead of it being #defined, it is defined as a value in an
enum?), then RTEXT_FILTER_VF wouldn't be set in the request. Of course
if RTEXT_FILTER_VF really doesn't exist, then the kernel should be
returning IFLA_VF_INFO without needing the extra flag in the request.

Hopefully something in here will point you at the source of your problem.