[libvirt] sr-iov ethernet VF is not assigned correctly to guest

[Scott]

On Mar 26, 2012, at 11:20 AM, Daniel P. Berrange wrote:

> On Mon, Mar 26, 2012 at 11:12:56AM -0400, Scott wrote:
>> Hi,  I'm hoping someone could possibly show me what I may be doing wrong here, and why I'm not seeing the expected results
>> 
>> Thanks in advance!
>> 
>> Server Information:HP Product Name: ProLiant DL165 G7
>> Ethernet Card Information: Intel 82599EB 10-Gigabit SFI/SFP+
>> uname: Linux 3.0.0-16-server x86_64 GNU/Linux
>> OS Release: Ubuntu 11.10
>> qemu-kvm version 0.14.1+noroms-0ubuntu6.2
>> 
>> First, here's a list of my VFs:
>> # ip link show:
>> 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP qlen 1000
>>   link/ether 00:1b:21:7d:70:16 brd ff:ff:ff:ff:ff:ff
>>   vf 0 MAC 02:22:25:00:00:00
>>   vf 1 MAC 02:22:25:00:00:01
>>   vf 2 MAC 02:22:25:00:00:02
>>   vf 3 MAC 02:22:25:00:00:03
>>   vf 4 MAC 02:22:25:00:00:04
>>   vf 5 MAC 02:22:25:00:00:05
>>   vf 6 MAC 02:22:25:00:00:06
>> 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN qlen 1000
>>   link/ether 00:1b:21:7d:70:16 brd ff:ff:ff:ff:ff:ff
>>   vf 0 MAC 02:22:25:01:00:00
>>   vf 1 MAC 02:22:25:01:00:01
>>   vf 2 MAC 02:22:25:01:00:02
>>   vf 3 MAC 02:22:25:01:00:03
>>   vf 4 MAC 02:22:25:01:00:04
>>   vf 5 MAC 02:22:25:01:00:05
>>   vf 6 MAC 02:22:25:01:00:06
>> 
>> # lspci |grep 05
>> 05:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>> 05:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>> 05:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:10.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:10.5 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:10.7 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:11.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:11.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:11.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:11.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:11.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 05:11.5 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>> 
>> # virsh nodedev-list |grep 05
>> pci_0000_05_00_0
>> pci_0000_05_00_1
> 
>> pci_0000_05_10_0
>> pci_0000_05_10_1
>> pci_0000_05_10_2
>> pci_0000_05_10_3
>> pci_0000_05_10_4
>> pci_0000_05_10_5
>> pci_0000_05_10_6
>> pci_0000_05_10_7
> 
>> pci_0000_05_11_0
>> pci_0000_05_11_1
>> pci_0000_05_11_2
>> pci_0000_05_11_3
>> pci_0000_05_11_4
>> pci_0000_05_11_5
>> 
>> 
>> 4 Now detach 05.10.0 and 05.11.0 from the host:
>> # virsh nodedev-dettach pci_0000_05_10_0
>> Device pci_0000_05_10_0 dettached
>> 
>> # virsh nodedev-dettach pci_0000_05_11_0
>> Device pci_0000_05_11_0 dettached
>> ip link:
>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
>>   link/ether 02:22:25:00:00:00 brd ff:ff:ff:ff:ff:ff
>> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
>>   link/ether 02:22:25:00:00:04 brd ff:ff:ff:ff:ff:ff
>> 
>> 
>> Expected Results:
>> GUEST-A should have the following assignment:
>> eth0 should = eth0-vf0 (bus:5, slot:10, fun:0), mac: 02:22:25:00:00:00
>> eth1 should = eth1-vf0 (bus:5, slot:11, fun:0), mac: 02:22:25:01:00:00
> 
> 
> I don't see where you check that  pci_0000_05_10_0 and pci_0000_05_11_0
> really do correspond to the virtual functions you want. ie where are
> you checking that they actually have the expected MAC addrs, before
> assigning them to the guest

Thank you for your response.  I had set the MAC addresses via:
for i in {0..6}; do ip link set eth0 vf $i mac 02:22:25:00:00:0$i; done
for i in {0..6}; do ip link set eth1 vf $i mac 02:22:25:01:00:0$i; done

then made sure they corresponded via ip link show (see output above)

Is there something else I should be doing/checking?

Thanks.