[libvirt-users] intel quad gigabit nic and pci passthrough
Laine Stump
laine at laine.org
Tue Jan 28 10:58:05 UTC 2014
On 01/25/2014 06:54 PM, Ivan Kabaivanov wrote:
> Hi all
>
> I have a very weird case of pci passthrough.
>
> I have a machine with 7 network interfaces, all of them intel. Four of them are on one quad giga ethernet device. If I manually unbind the devices and allow qemu to use them, with intel IOMMU working, everything works like a charm. Here's how I do it manually:
>
> root at kybrat (x86_64) ~]$ lspci -nn | grep net
> 00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 06)
> 02:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) [8086:107d] (rev 06)
> 05:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
> 05:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
> 06:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
> 06:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller [8086:10a4] (rev 06)
> 0c:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) [8086:107d] (rev 06)
>
>
> echo "8086 10a4" > /sys/bus/pci/drivers/pci-stub/new_id
> echo "8086 107d" > /sys/bus/pci/drivers/pci-stub/new_id
> echo "0000:05:00.0" > /sys/bus/pci/devices/0000\:05\:00.0/driver/unbind
> echo "0000:05:00.0" > /sys/bus/pci/drivers/pci-stub/bind
> echo "0000:05:00.1" > /sys/bus/pci/devices/0000\:05\:00.1/driver/unbind
> echo "0000:05:00.1" > /sys/bus/pci/drivers/pci-stub/bind
> echo "0000:06:00.0" > /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind
> echo "0000:06:00.0" > /sys/bus/pci/drivers/pci-stub/bind
> echo "0000:06:00.1" > /sys/bus/pci/devices/0000\:06\:00.1/driver/unbind
> echo "0000:06:00.1" > /sys/bus/pci/drivers/pci-stub/bind
> echo "0000:0c:00.0" > /sys/bus/pci/devices/0000\:0c\:00.0/driver/unbind
> echo "0000:0c:00.0" > /sys/bus/pci/drivers/pci-stub/bind
>
> qemu-system-i386 -enable-kvm \
> -m 2048 \
> -cpu host \
> -machine pc-q35-1.6 \
> -drive file=/media/virtual/krym.qcow2,if=virtio \
> -net none \
> -device pci-assign,host=06:00.1,rombar=0 \
> -device pci-assign,host=05:00.0,rombar=0 \
> -device pci-assign,host=06:00.0,rombar=0 \
> -device pci-assign,host=05:00.1,rombar=0 \
> -device pci-assign,host=0c:00.0,rombar=0 \
> -cdrom /media/virtual/asg-9.107-33.1.iso \
> -boot c \
> -vnc :0 \
> -runas virtual \
> -monitor unix:/media/virtual/control/krym/monitor,server,nowait \
> -rtc base=localtime,clock=host,driftfix=none
>
> This starts qemu and pci passthrough works nicely.
>
> Now, if I try to do the same thing through libvirt, virt-install doesn't like the quad ethernet device pci passthrough and I get the error below:
>
> virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_06_00_1 --host-device=pci_0000_05_00_0 --host-device=pci_0000_06_00_0 --host-device=pci_0000_05_00_1 --host-device=pci_0000_0c_00_0
>
> Starting install...
> ERROR Requested operation is not valid: PCI device 0000:06:00.1 is not assignable
> Domain installation does not appear to have been successful.
> If it was, you can restart your domain by running:
> virsh --connect qemu:///system start krym
> otherwise, please restart your installation.
More information is needed to make a guess at the source of the problem:
1) What distro/version of Linux are you running.
2) What version of libvirt?
3) After the failed virt-install command, if the new domain is still
defined, can you provide the output of:
virsh dumpxml $domain
(if virt-install is automatically undefining the failed domain, then you
can get the xml for the defined domain by appending "--print-xml" to the
virt-install commandline)
4) Also, send the last several lines of
/var/log/libvirt/qemu/$domain.log, in particular the qemu commandline
that was executed to start the domain, and any subsequent error messages.
(Just one possibility - since you are successfully using the q35
machinetype, it's highly likely that your OS is new enough that it
supports vfio for device passthrough; vfio is more restrictive about
what devices can be assigned to a guest, sometimes requiring seemingly
unrelated devices to be assigned to the same guest, and the most recent
libvirts will prefer vfio over legacy kvm device assignment when
available. So it's possible that your manual running of qemu-kvm works
because you are forcing use of legacy kvm assignment, and when libvirt
attempts to use vfio it is failing. This will be made apparent by the
information I requested above.
>
>
> If I try to use just one NIC (any NIC, as long as it's not the quad NIC), things do work:
>
> virt-install --name=krym --ram=2048 --vcpus=1 --cpu=host --import --os-type=linux --disk path=/media/virtual/krym.qcow2,format=qcow2,bus=virtio --nonetworks --graphics vnc,port=5900 --virt-type=kvm --machine=q35 --accelerate --host-device=pci_0000_0c_00_0
>
> Starting install...
> Creating domain... | 0 B 00:00:01
> Cannot open display:
> Run 'virt-viewer --help' to see a full list of available command line options
> Domain creation completed. You can restart your domain by running:
> virsh --connect qemu:///system start krym
>
>
>
> So I guess libvirt has some problem with the intel quad NIC.
>
> I will appreciate any help.
>
> Thanks,
> IvanK.
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users at redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
>
>
More information about the libvirt-users
mailing list