[vfio-users] GPU Passthrough, 2 GPUs to different virtual machines

[Torbjorn Jansson]

On 2017-12-31 13:31, Sascha Fröhlich wrote:
> 2017-12-31 13:21 GMT+01:00 Torbjorn Jansson <
> torbjorn.jansson at mbox200.swipnet.se>:
>> On 2017-12-31 12:25, Sascha Fröhlich wrote:
>>
>>> Hi,
>>>
>>> I am trying to get the following setup with a Xeon E5/ASUS X99-A II to
>>> work:
>>>
>>> First slot:      GTX 1060     -> VM 1
>>> Second slot: GTX 1070 -> VM 2
>>> Third slot:     GT 710         -> Host
>>>
>>> This is the latest ArchLinux. I had to use the propietary NVIDIA drivers
>>> and an additional Xorg configuration for the host GPU, because nouveau
>>> would flicker and crash. No need to blacklist anything.
>>>
>>> IOMMU and VFIO works fine though, lspci -nn -k reports the two GTX GPUs
>>> using vfio-pci:
>>> https://gist.github.com/anonymous/89ebd0e18464cef3c5dde6897c39eb5f
>>>
>>>
>> did you make sure that the proprietary nvidia driver is never
>> attached/loaded for the 2 cards you intend to forward to a vm?
>> see: http://vfio.blogspot.se/2015/05/vfio-gpu-how-to-series-part-
>> 3-host.html
>>
>>
> 
> I am loading the appropiate vfio modules at boot time before the nvidia
> module and I use efifb=off as kernel parameter. lspci -nnk states that the
> 2 cards are correctly claimed by vfio-pci.
>
> Additionally, I tried to use the driver_override approach mentioned in the
> blog post, but to no avail.
>

just loading vfio modules is not enough.
you also need to make sure vfio-pci is bound to your cards in question directly 
at boot before nvidia module or anything else grabs them.
this is done via module parameter to vfio-pci