[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [vfio-users] passthrough usb controler pci-e card ?

I have a controller that I pass through for some USB devices that don't work in linux and I just bind it out early because it's easier than binding out the drivers that attach to the devices on the controller and then don't work and won't unbind. It sounds like you may have a similar problem and looking at what you described I'd say your boot solution may not work because you don't have vfio-pci in your initramfs. Add the following to your dracut.conf but bear in mind I don't use fedora or dracut so you might need to find the right syntax/file. Hopefully your init script will work after that - stick some messages in dmesg with it (command is logger on my system).

drivers+="vfio-pci vfio vfio-virqfd vfio-iommu-type1"

As an alternative you can try the pci-stub module which is probably a good idea if your controller is misbehaving and won't unbind post-boot. It is simpler and won't power save the controller plus I think in fedora it might be compiled into the kernel by default on some versions in which case you can avoid all the above and definitely beat the xhci drivers to binding to the controller by adding the following to your kernel command line:


There's a tonne of other ways to do this but those should work and keep working. Other things to check are that any drives aren't auto-mounted before you try the post-boot unbind and that any devices on the usb bus unbind properly. I'd like to think unbinding the host driver would shut them all down - but that doesn't always happen and you say it works after a clean no-device reboot plus things have their own power source and generic drivers don't work with everything. One way to do this is to find the directory for your hub in sysfs and unbind them one by one then rebind them and see if they play ball (don't forget lsusb -vt).


The final thing to check thoroughly is that your controller is suitable for passing through. It should obviously have it's own IOMMU group (or be grouped only with things your passing through), but double check with lspci -tv that it's not under a root bridge with something else and if it is move the card or switch off the something else. Lots of things can break the isolation like shared IRQs or PCI power saving/AER/Hotplug and moving cards around may help .. but your controller/mboard may just not isolate themselves sufficiently to passthrough without silly constraints.

If it doesn't just work I would consider your options though. Most of my built in USB controllers have hard wired IO ports so would be asking for trouble on pass through (of the intermittent variety) plus if it has the drivers Linux does a much better job of scheduling/hangs/resets/ACPI/etc. than windows or an isolated linux VM plus usb passthrough (i.e. -usb -usb_host .... for qemu) costs basically nothing important as USB drives don't have the throughput (unless you've got some prototype full speed USB 3.1 stuff lying around). I've had no troubles using passthrough with USB-Sticks / drives - I even re-installed the firmware on a USB drive that I was passing through. I wouldn't try that at home .. but installing windows on unviolated bare metal risks the entire system as far as I'm concerned.

On 25/03/16 17:23, thibaut noah wrote:
Bumping this as i still have some weird behaviour.
Basically whether i unbind the device to bind it to vfio or let it managed in my libvirt xml it is the same.
Sometimes the vm will launch and sometimes i find myself hanging on the tianocore screen.
Pretty sure it comes from the usb devices plugged into the card as stated above, problem is i cannot plug and unplug everything i boot this is ridiculous.
Also having my external hard drive connected to the usb card will prevent me to boot fedora completly, so blacklisting the device for the os before the linux usb driver is loaded would be great, but since it is kinda different than blacklisting the graphic card i failed to do it, any clue?

2016-01-28 20:16 GMT+01:00 Frederick N. Brier <fnbrier gmail com>:
I finally got my VM working with PCI passthrough for an nVidia card and a PCI-E USB controller card.  So far, no problems.  Thank you Alex for all your wonderful blog and forum posts.  The use of TightVNC was invaluable.  My last niggling problem is automating two commands:

echo "0000:04:00.0" > /sys/bus/pci/devices/0000:04:00.0/driver/unbind
echo 0x1912 0x0014 > /sys/bus/pci/drivers/vfio-pci/new_id

I currently type them when the host system boots.  I tried putting them in a /sbin/vfio-pci-override-usb.sh script referenced in the /etc/dracut.conf.d/local.conf file with the line:


However, the kernel drive in use is still xhci_hcd, not vfio-pci, as it is after invoking the above two commands.  Where should these commands go to get executed at boot?  Thank you.


On 01/19/2016 03:21 AM, thibaut noah wrote:
Update note, i found what prevent my card from running, basically i need to reboot without anything plug into the card and then i can unbind and rebind the card, what that strange behaviour?

2016-01-19 11:35 GMT+01:00 thibaut noah <thibaut noah gmail com>:
Bumping this, running virsh nodedev-detach pci_0000_xx_yy_z'  (with proper numbers) and/or having managed=yes in the xml file changes nothing (actually i had this already), ovmf still hangs waiting for i have no idea what.
Cannot run by unbinding the device through script either, seems that i was lucky, or maybe i did something that i forgot.
Tried to add nodedev-detach to modprobe but it seems that i did not do it in the proper way so it wasn't working

2016-01-16 11:30 GMT+01:00 thibaut noah <thibaut noah gmail com>:
Didn't know libvirt was capable of unbinding devices on its own, good to know, i'm gonna try this and if i manage to make it work i don't have any reason to bother myself more with this. (note that i don't use virt-manager since you advise me to use libvirt directly)
Though the usb card will only be use by the vm, i have more than enough usb ports on my backpanel.

Tried the gpu method by adding the id of the device in modprobe.d after gpu's ids but it didn't work.

I paid it 50euros :(
Thanks for the explanations alex

2016-01-15 18:59 GMT+01:00 Alex Williamson <alex williamson redhat com>:
A couple comments, first, boot time pre-binding to vfio-pci is really
only necessary for devices where the native host drivers behave poorly
if you take the device away from them later.  This is why we do it for
GPUs and their companion sound device, host GPU drivers don't like to
give up the device, it plays poorly with any sort of graphics on the
host, and sequestering the audio device prevents host tools from
getting confused (and there are some bugs in the audio driver limiting
number of attach/detach cycles iirc).

For anything else, you can dynamically unbind the device from the host
driver, bind it to vfio while the VM is running, and give it back to
the host on shutdown.  libvirt will do this automatically for you if
your XML sets managed='yes' for the <hostdev> device.  This is the default, so if you use virt-manager to add the device, just select Add Hardware -> PCI Host Device -> select device -> Finish.  Done.  If for some reason you don't want the device flopping back and forth between host and guest, just run 'virsh nodedev-detach pci_0000_xx_yy_z' at bootup where xx_yy_z is the PCI bus (xx), device (yy), and function (z) numbers, the same as in lspci.  You can adopt some of the GPU methods for doing this if you want it to happen earlier as well, there are lots of ways to do this with modprobe.d (install options, softdep, etc..)

Finally, yes I've seen OVMF hang with some crappy USB controllers.  I'm
not sure if it's dependent on the devices attached or the controller
itself, but cheaper isn't always better when it comes to selecting
devices to use with device assignment.  Thanks,


vfio-users mailing list
vfio-users redhat com

vfio-users mailing list
vfio-users redhat com

vfio-users mailing list
vfio-users redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]