[vfio-users] two nvidia cards, one for the host with nvidia module

[ddio at jr0.de]

Am 2016-03-07 16:55, schrieb Alex Williamson:
> nvidia.ko does not support dynamic unbinding of devices.  I've filed a
> bug with them, they closed it wontfix.  You need to follow the guides
> to have the VM graphics card bound to pci-stub or vfio-pci directly
> rather than letting it bind to nvidia.  See for instance
> http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-3-host.html

Forgot to send it also to the mailinglist...

Ok sry, after double reading it, I understood that this pci ids are 
generic for any nvidia or amd cards, so I'm not allowed to use them 
because I need one nvidia card for my host.

I set up dracut with add_drivers+="vfio vfio_iommu_type1 vfio_pci 
vfio_virqfd" and vfio-pci with options vfio-pci ids=10de:1401,10de:0fba

because of:

lspci -knn

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK106 
[GeForce GTX 650 Ti] [10de:11c6] (rev a1)
     Subsystem: ZOTAC International (MCO) Ltd. GK106 [GeForce GTX 650 Ti] 
[19da:5263]
     Kernel driver in use: nvidia
     Kernel modules: nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation GK106 HDMI Audio 
Controller [10de:0e0b] (rev a1)
     Subsystem: ZOTAC International (MCO) Ltd. GK106 HDMI Audio 
Controller [19da:5263]
     Kernel driver in use: snd_hda_intel
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM206 
[GeForce GTX 960] [10de:1401] (rev a1)
     Subsystem: eVga.com. Corp. GM206 [GeForce GTX 960] [3842:3967]
     Kernel driver in use: vfio-pci
     Kernel modules: nvidia
02:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fba] (rev 
a1)
     Subsystem: eVga.com. Corp. Device [3842:3967]
     Kernel driver in use: vfio-pci

That way it only affects my GTX960.

I did only use vfio-pci because thats available for my kernel (4.1.15) 
and therefor I don't need pci stubs and is not recommended as vfio-pci 
is the newer way.

Any chance a newer kernel would fix my problem?

But as you can see for Kernel module is still nvidia put in :/. Did I do 
sth wrong? Or perhaps this is normal and no problem because of Kernel 
driver in use: vfio-pci is correct?

Is the nvidia module per default in dracut ( so I would need to remove 
that from the initrd)?

In the list generated by dracut --list-modules its not included:

bash
bootchart
dash
systemd
systemd-bootchart
systemd-initrd
caps
systemd-networkd
modsign
rescue
watchdog
busybox
i18n
convertfs
network
ifcfg
url-lib
drm
gensplash
plymouth
cms
btrfs
crypt
dm
dmraid
dmsquash-live
kernel-modules
kernel-network-modules
livenet
lvm
mdraid
multipath
qemu
qemu-net
crypt-gpg
crypt-loop
cifs
dasd
dasd_mod
dasd_rules
fcoe-uefi
fstab-sys
iscsi
nbd
nfs
resume
rootfs-block
ssh-client
terminfo
udev-rules
virtfs
zfcp
zfcp_rules
znet
biosdevname
dracut-systemd
ecryptfs
pollcdrom
syslog
usrmount
base
fs-lib
img-lib
shutdown
uefi-lib