[vfio-users] GTX 970 Passthrough from Arch Linux (Manjaro) Host to Arch Linux Guest

Wed May 24 00:33:44 UTC 2017

Hello and thank you for all of your work.

Distro version: 4.9.28-1-MANJARO - 4.9.28-1 kernel

QEMU version: QEMU emulator version 2.9.0

libvirtd version: libvirtd (libvirt) 3.2.0  

virt-manager version: 1.4.1

guest vm: arch linux 2017.05.01 - 4.10.13 kernel

I am a noob to the community (or even the idea of community) "long time mac
user" but am loving linux so far.

Recently I have been trying to get GPU passthrough working for a Linux guest.
Previously I have only been successful passing through gpu to macOS guests;
MacOS 10.12.5, MacOS 10.12.4, MacOS 10.11.4 and I have never attempted
windows.

For the longest time I couldn't get drivers installed so this last time I
installed Arch Linux on metal in the aspired configuration then passed the
installed disk to the vm in virt-manager by editing the xml manually.

The vm outputs to the gpu and I get a display on my monitor but after grub
nothing continues on the screen.

I have modified versions of the xml from ideas in this mainling list and
several online source quite a few times all with similar lackluster results.
That being said most examples that exist seem geared toward windows.

Hopefully it goes without saying, but starting the vm without the gpu works
fine.

below are copy/pasted lines from the xml and last log:

Thank you in advance for any guidance.

Allen Dial

_domain.xml_________________________

<domain type='kvm'>  
 <name>arch-anywhere-nv</name>  
 <uuid>3f1e4dbc-e47f-4dbf-9d21-984efbb2f9dc</uuid>  
 <title>arch-anywhere-nv</title>  
 <memory unit='KiB'>16777216</memory>  
 <currentMemory unit='KiB'>16777216</currentMemory>  
 <vcpu placement='static'>4</vcpu>  
 <os>  
   <type arch='x86_64' machine='pc-q35-2.9'>hvm</type>  
   <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>  
   <nvram>/var/lib/libvirt/qemu/nvram/arch-anywhere-nv_VARS.fd</nvram>  
   <boot dev='cdrom'/>  
   <boot dev='hd'/>  
 </os>  
 <features>  
   <acpi/>  
   <kvm>  
     <hidden state='on'/>  
   </kvm>  
 </features>  
 <cpu mode='custom' match='exact' check='partial'>  
   <model fallback='allow'>Haswell-noTSX</model>  
 </cpu>  
 <clock offset='utc'>  
   <timer name='rtc' tickpolicy='catchup'/>  
   <timer name='pit' tickpolicy='delay'/>  
   <timer name='hpet' present='no'/>  
 </clock>  
 <on_poweroff>destroy</on_poweroff>  
 <on_reboot>destroy</on_reboot>  
 <on_crash>destroy</on_crash>  
 <pm>  
   <suspend-to-mem enabled='no'/>  
   <suspend-to-disk enabled='no'/>  
 </pm>  
 <devices>  
   <emulator>/usr/sbin/qemu-system-x86_64</emulator>  
   <disk type='block' device='disk'>  
     <driver name='qemu' type='raw'/>  
     <source dev='/dev/disk/by-id/ata-SanDisk_SDSSDA240G_170735467315'/>  
     <target dev='vdd' bus='virtio'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>  
   </disk>  
   <disk type='file' device='cdrom'>  
     <driver name='qemu' type='raw'/>  
     <target dev='sdb' bus='sata'/>  
     <readonly/>  
     <address type='drive' controller='0' bus='0' target='0' unit='1'/>  
   </disk>  
   <controller type='usb' index='0' model='ich9-ehci1'>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/>  
   </controller>  
   <controller type='usb' index='0' model='ich9-uhci1'>  
     <master startport='0'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/>  
   </controller>  
   <controller type='usb' index='0' model='ich9-uhci2'>  
     <master startport='2'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/>  
   </controller>  
   <controller type='usb' index='0' model='ich9-uhci3'>  
     <master startport='4'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/>  
   </controller>  
   <controller type='sata' index='0'>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>  
   </controller>  
   <controller type='pci' index='0' model='pcie-root'/>  
   <controller type='pci' index='1' model='dmi-to-pci-bridge'>  
     <model name='i82801b11-bridge'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>  
   </controller>  
   <controller type='pci' index='2' model='pci-bridge'>  
     <model name='pci-bridge'/>  
     <target chassisNr='2'/>  
     <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>  
   </controller>  
   <controller type='pci' index='3' model='pcie-root-port'>  
     <model name='pcie-root-port'/>  
     <target chassis='3' port='0x8'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>  
   </controller>  
   <controller type='pci' index='4' model='pcie-root-port'>  
     <model name='pcie-root-port'/>  
     <target chassis='4' port='0x9'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>  
   </controller>  
   <controller type='pci' index='5' model='pcie-root-port'>  
     <model name='pcie-root-port'/>  
     <target chassis='5' port='0xa'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>  
   </controller>  
   <controller type='pci' index='6' model='pcie-root-port'>  
     <model name='pcie-root-port'/>  
     <target chassis='6' port='0xb'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>  
   </controller>  
   <controller type='pci' index='7' model='pcie-root-port'>  
     <model name='pcie-root-port'/>  
     <target chassis='7' port='0xc'/>  
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>  
   </controller>  
   <controller type='virtio-serial' index='0'>  
     <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>  
   </controller>  
   <serial type='pty'>  
     <target port='0'/>  
   </serial>  
   <console type='pty'>  
     <target type='serial' port='0'/>  
   </console>  
   <channel type='spicevmc'>  
     <target type='virtio' name='com.redhat.spice.0'/>  
     <address type='virtio-serial' controller='0' bus='0' port='1'/>  
   </channel>  
   <input type='mouse' bus='ps2'/>  
   <input type='keyboard' bus='ps2'/>  
   <sound model='ich6'>  
     <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>  
   </sound>  
   <hostdev mode='subsystem' type='usb' managed='yes'>  
     <source>  
       <vendor id='0x05ac'/>  
       <product id='0x0304'/>  
     </source>  
     <address type='usb' bus='0' port='2'/>  
   </hostdev>  
   <hostdev mode='subsystem' type='usb' managed='yes'>  
     <source>  
       <vendor id='0x05ac'/>  
       <product id='0x024f'/>  
     </source>  
     <address type='usb' bus='0' port='3'/>  
   </hostdev>  
   <hostdev mode='subsystem' type='pci' managed='yes'>  
     <source>  
       <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>  
     </source>  
     <rom bar='on'/>  
     <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>  
   </hostdev>  
   <hostdev mode='subsystem' type='pci' managed='yes'>  
     <source>  
       <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/>  
     </source>  
     <rom bar='on'/>  
     <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>  
   </hostdev>  
   <hostdev mode='subsystem' type='usb' managed='yes'>  
     <source>  
       <vendor id='0x13b1'/>  
       <product id='0x0041'/>  
     </source>  
     <address type='usb' bus='0' port='1'/>  
   </hostdev>  
   <redirdev bus='usb' type='spicevmc'>  
     <address type='usb' bus='0' port='4'/>  
   </redirdev>  
   <redirdev bus='usb' type='spicevmc'>  
     <address type='usb' bus='0' port='5'/>  
   </redirdev>  
   <memballoon model='virtio'>  
     <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>  
   </memballoon>  
 </devices>  
</domain>  

_log_________________________  

2017-05-23 22:34:33.400+0000: starting up libvirt version: 3.2.0, qemu
version: 2.9.0, hostname: prototype.filmtek-cloud.intranet  
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
QEMU_AUDIO_DRV=none /usr/sbin/qemu-system-x86_64 -name guest=arch-anywhere-nv
,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-arch-
anywhere-nv/master-key.aes -machine pc-q35-2.9,accel=kvm,usb=off,dump-guest-
core=off -cpu Haswell-noTSX,kvm=off -drive
file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on
-drive file=/var/lib/libvirt/qemu/nvram/arch-anywhere-
nv_VARS.fd,if=pflash,format=raw,unit=1 -m 16384 -realtime mlock=off -smp
4,sockets=4,cores=1,threads=1 -uuid 3f1e4dbc-e47f-4dbf-9d21-984efbb2f9dc
-display none -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-arch-anywhere-
nv/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control
-rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet
-no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot
strict=on -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-
bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 -device pcie-root-
port,port=0x8,chassis=3,id=pci.3,bus=pcie.0,multifunction=on,addr=0x1 -device
pcie-root-port,port=0x9,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x1 -device
pcie-root-port,port=0xa,chassis=5,id=pci.5,bus=pcie.0,addr=0x1.0x2 -device
pcie-root-port,port=0xb,chassis=6,id=pci.6,bus=pcie.0,addr=0x1.0x3 -device
pcie-root-port,port=0xc,chassis=7,id=pci.7,bus=pcie.0,addr=0x1.0x4 -device
ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 -device ich9-usb-
uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2
-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -drive
file=/dev/disk/by-id/ata-SanDisk_SDSSDA240G_170735467315,format=raw,if=none,id
=drive-virtio-disk3 -device virtio-blk-pci,scsi=off,bus=pcie.0,addr=0x9,drive
=drive-virtio-disk3,id=virtio-disk3,bootindex=2 -drive if=none,media=cdrom,id
=drive-sata0-0-1,readonly=on -device ide-cd,bus=ide.1,drive=drive-
sata0-0-1,id=sata0-0-1,bootindex=1 -chardev pty,id=charserial0 -device isa-
serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-
serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-device intel-hda,id=sound0,bus=pci.2,addr=0x1 -device hda-
duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
spicevmc,id=charredir0,name=usbredir -device usb-
redir,chardev=charredir0,id=redir0,bus=usb.0,port=4 -chardev
spicevmc,id=charredir1,name=usbredir -device usb-
redir,chardev=charredir1,id=redir1,bus=usb.0,port=5 -device usb-
host,hostbus=3,hostaddr=5,id=hostdev0,bus=usb.0,port=2 -device usb-
host,hostbus=3,hostaddr=10,id=hostdev1,bus=usb.0,port=3 -device vfio-
pci,host=07:00.0,id=hostdev2,bus=pci.4,addr=0x0,rombar=1 -device vfio-
pci,host=07:00.1,id=hostdev3,bus=pci.5,addr=0x0,rombar=1 -device virtio-
balloon-pci,id=balloon0,bus=pci.6,addr=0x0 -msg timestamp=on  
2017-05-23 22:34:33.400+0000: Domain id=3 is tainted: high-privileges  
2017-05-23T22:34:33.443075Z qemu-system-x86_64: -chardev pty,id=charserial0:
char device redirected to /dev/pts/2 (label charserial0)  
2017-05-23T22:36:32.836901Z qemu-system-x86_64: terminating on signal 15 from
pid 14129 (/usr/bin/libvirtd)  
2017-05-23 22:36:34.437+0000: shutting down, reason=destroyed  
2017-05-23 23:22:26.933+0000: starting up libvirt version: 3.2.0, qemu
version: 2.9.0, hostname: prototype.filmtek-cloud.intranet  
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
QEMU_AUDIO_DRV=none /usr/sbin/qemu-system-x86_64 -name guest=arch-anywhere-nv
,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-arch-
anywhere-nv/master-key.aes -machine pc-q35-2.9,accel=kvm,usb=off,dump-guest-
core=off -cpu Haswell-noTSX,kvm=off -drive
file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on
-drive file=/var/lib/libvirt/qemu/nvram/arch-anywhere-
nv_VARS.fd,if=pflash,format=raw,unit=1 -m 16384 -realtime mlock=off -smp
4,sockets=4,cores=1,threads=1 -uuid 3f1e4dbc-e47f-4dbf-9d21-984efbb2f9dc
-display none -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-arch-anywhere-
nv/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control
-rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet
-no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot
strict=on -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-
bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 -device pcie-root-
port,port=0x8,chassis=3,id=pci.3,bus=pcie.0,multifunction=on,addr=0x1 -device
pcie-root-port,port=0x9,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x1 -device
pcie-root-port,port=0xa,chassis=5,id=pci.5,bus=pcie.0,addr=0x1.0x2 -device
pcie-root-port,port=0xb,chassis=6,id=pci.6,bus=pcie.0,addr=0x1.0x3 -device
pcie-root-port,port=0xc,chassis=7,id=pci.7,bus=pcie.0,addr=0x1.0x4 -device
ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 -device ich9-usb-
uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2
-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -drive
file=/dev/disk/by-id/ata-SanDisk_SDSSDA240G_170735467315,format=raw,if=none,id
=drive-virtio-disk3 -device virtio-blk-pci,scsi=off,bus=pcie.0,addr=0x9,drive
=drive-virtio-disk3,id=virtio-disk3,bootindex=2 -drive if=none,media=cdrom,id
=drive-sata0-0-1,readonly=on -device ide-cd,bus=ide.1,drive=drive-
sata0-0-1,id=sata0-0-1,bootindex=1 -chardev pty,id=charserial0 -device isa-
serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-
serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-device intel-hda,id=sound0,bus=pci.2,addr=0x1 -device hda-
duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
spicevmc,id=charredir0,name=usbredir -device usb-
redir,chardev=charredir0,id=redir0,bus=usb.0,port=4 -chardev
spicevmc,id=charredir1,name=usbredir -device usb-
redir,chardev=charredir1,id=redir1,bus=usb.0,port=5 -device usb-
host,hostbus=3,hostaddr=5,id=hostdev0,bus=usb.0,port=2 -device usb-
host,hostbus=3,hostaddr=10,id=hostdev1,bus=usb.0,port=3 -device vfio-
pci,host=07:00.0,id=hostdev2,bus=pci.4,addr=0x0,rombar=1 -device vfio-
pci,host=07:00.1,id=hostdev3,bus=pci.5,addr=0x0,rombar=1 -device usb-
host,hostbus=3,hostaddr=12,id=hostdev4,bus=usb.0,port=1 -device virtio-
balloon-pci,id=balloon0,bus=pci.6,addr=0x0 -msg timestamp=on  
2017-05-23 23:22:26.933+0000: Domain id=4 is tainted: high-privileges  
2017-05-23T23:22:26.979538Z qemu-system-x86_64: -chardev pty,id=charserial0:
char device redirected to /dev/pts/2 (label charserial0)  

![](https://link.nylas.com/open/1m3veovga7nl7tepk1he21tjo/local-41c73afc-
d16c?r=dmZpby11c2Vyc0ByZWRoYXQuY29t)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20170524/415d9eb7/attachment.htm>