[vfio-users] GTX 970 Passthrough from Arch Linux (Manjaro) Host to Arch Linux Guest
Allen Dial
adial at mac.com
Thu May 25 00:38:11 UTC 2017
Update:
Thank you Zir Blazer and Anthanasios Tasoglou
I was able to get it working today in qemu with the following that I cobbled
together from <http://turlucode.com/qemu-kvm-on-arch-linux-guide> and right
here <https://www.redhat.com/archives/vfio-users/2016-May/msg00010.html>
Tomorrow I will try to figure out why it is so slow. Assuming I can get it, I
will then try and convert it to domain.xml afterward and post the results here
in case someone else could use them.
I'm sure it is me, but it's crazy how much more difficult gpu passthrough from
linux host to linux guest has been to achieve than linux host to mac guest.
Thanks again
Allen
sudo qemu-system-x86_64 \
-enable-kvm -m 32764 \
-cpu host,kvm=off \
-smp 8,sockets=1,cores=8,threads=1 \
-machine pc-q35-2.5,accel=kvm \
-vga none \
-device ioh3420,chassis=1,bus=pcie.0,id=pcie.1 \
-device vfio-pci,multifunction=true,bus=pcie.1,addr=00.0,host=07:00.0 \
-device vfio-pci,bus=pcie.1,addr=00.1,host=07:00.1 \
-device vfio-pci,host=04:00.0,bus=pcie.0 \
-drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF.fd \
-drive if=pflash,format=raw,file=/var/lib/libvirt/qemu/nvram/arch-anywhere-nv_VARS.fd \
-hda /dev/sdc \
-usb -usbdevice host:05ac:0304 \
-usb -usbdevice host:05ac:024f
On May 23 2017, at 5:33 pm, Allen Dial <adial at mac.com> wrote:
> Hello and thank you for all of your work.
>
>
>
> Distro version: 4.9.28-1-MANJARO - 4.9.28-1 kernel
>
> QEMU version: QEMU emulator version 2.9.0
>
> libvirtd version: libvirtd (libvirt) 3.2.0
>
> virt-manager version: 1.4.1
>
> guest vm: arch linux 2017.05.01 - 4.10.13 kernel
>
>
>
> I am a noob to the community (or even the idea of community) "long time mac
user" but am loving linux so far.
>
>
>
> Recently I have been trying to get GPU passthrough working for a Linux
guest. Previously I have only been successful passing through gpu to macOS
guests; MacOS 10.12.5, MacOS 10.12.4, MacOS 10.11.4 and I have never attempted
windows.
>
>
>
> For the longest time I couldn't get drivers installed so this last time I
installed Arch Linux on metal in the aspired configuration then passed the
installed disk to the vm in virt-manager by editing the xml manually.
>
>
>
> The vm outputs to the gpu and I get a display on my monitor but after grub
nothing continues on the screen.
>
>
>
> I have modified versions of the xml from ideas in this mainling list and
several online source quite a few times all with similar lackluster results.
That being said most examples that exist seem geared toward windows.
>
>
>
> Hopefully it goes without saying, but starting the vm without the gpu works
fine.
>
>
>
> below are copy/pasted lines from the xml and last log:
>
>
>
> Thank you in advance for any guidance.
>
>
>
> Allen Dial
>
>
>
> _domain.xml_________________________
>
>
>
> <!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
virsh edit arch-anywhere-nv
or other application using the libvirt API.
\-->
<domain type='kvm'>
<name>arch-anywhere-nv</name>
<uuid>3f1e4dbc-e47f-4dbf-9d21-984efbb2f9dc</uuid>
<title>arch-anywhere-nv</title>
<memory unit='KiB'>16777216</memory>
<currentMemory unit='KiB'>16777216</currentMemory>
<vcpu placement='static'>4</vcpu>
<os>
<type arch='x86_64' machine='pc-q35-2.9'>hvm</type>
<loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
<nvram>/var/lib/libvirt/qemu/nvram/arch-anywhere-nv_VARS.fd</nvram>
<boot dev='cdrom'/>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<kvm>
<hidden state='on'/>
</kvm>
</features>
<cpu mode='custom' match='exact' check='partial'>
<model fallback='allow'>Haswell-noTSX</model>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>destroy</on_reboot>
<on_crash>destroy</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/sbin/qemu-system-x86_64</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw'/>
<source dev='/dev/disk/by-id/ata-SanDisk_SDSSDA240G_170735467315'/>
<target dev='vdd' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<target dev='sdb' bus='sata'/>
<readonly/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/>
</controller>
<controller type='sata' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1' model='dmi-to-pci-bridge'>
<model name='i82801b11-bridge'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<model name='pci-bridge'/>
<target chassisNr='2'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0x8'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0x9'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0xa'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='6' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0xb'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
</controller>
<controller type='pci' index='7' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='7' port='0xc'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
</controller>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</controller>
<serial type='pty'>
<target port='0'/>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='spicevmc'>
<target type='virtio' name='com.redhat.spice.0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<sound model='ich6'>
<address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
</sound>
<hostdev mode='subsystem' type='usb' managed='yes'>
<source>
<vendor id='0x05ac'/>
<product id='0x0304'/>
</source>
<address type='usb' bus='0' port='2'/>
</hostdev>
<hostdev mode='subsystem' type='usb' managed='yes'>
<source>
<vendor id='0x05ac'/>
<product id='0x024f'/>
</source>
<address type='usb' bus='0' port='3'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
</source>
<rom bar='on'/>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/>
</source>
<rom bar='on'/>
<address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='usb' managed='yes'>
<source>
<vendor id='0x13b1'/>
<product id='0x0041'/>
</source>
<address type='usb' bus='0' port='1'/>
</hostdev>
<redirdev bus='usb' type='spicevmc'>
<address type='usb' bus='0' port='4'/>
</redirdev>
<redirdev bus='usb' type='spicevmc'>
<address type='usb' bus='0' port='5'/>
</redirdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</memballoon>
</devices>
</domain>
>
>
>
> _log_________________________
>
>
>
> 2017-05-23 22:34:33.400+0000: starting up libvirt version: 3.2.0, qemu
version: 2.9.0, hostname: prototype.filmtek-cloud.intranet
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
QEMU_AUDIO_DRV=none /usr/sbin/qemu-system-x86_64 -name guest=arch-anywhere-nv
,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-arch-
anywhere-nv/master-key.aes -machine pc-q35-2.9,accel=kvm,usb=off,dump-guest-
core=off -cpu Haswell-noTSX,kvm=off -drive
file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on
-drive file=/var/lib/libvirt/qemu/nvram/arch-anywhere-
nv_VARS.fd,if=pflash,format=raw,unit=1 -m 16384 -realtime mlock=off -smp
4,sockets=4,cores=1,threads=1 -uuid 3f1e4dbc-e47f-4dbf-9d21-984efbb2f9dc
-display none -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-arch-anywhere-
nv/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control
-rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet
-no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot
strict=on -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-
bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 -device pcie-root-
port,port=0x8,chassis=3,id=pci.3,bus=pcie.0,multifunction=on,addr=0x1 -device
pcie-root-port,port=0x9,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x1 -device
pcie-root-port,port=0xa,chassis=5,id=pci.5,bus=pcie.0,addr=0x1.0x2 -device
pcie-root-port,port=0xb,chassis=6,id=pci.6,bus=pcie.0,addr=0x1.0x3 -device
pcie-root-port,port=0xc,chassis=7,id=pci.7,bus=pcie.0,addr=0x1.0x4 -device
ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 -device ich9-usb-
uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2
-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -drive
file=/dev/disk/by-id/ata-SanDisk_SDSSDA240G_170735467315,format=raw,if=none,id
=drive-virtio-disk3 -device virtio-blk-pci,scsi=off,bus=pcie.0,addr=0x9,drive
=drive-virtio-disk3,id=virtio-disk3,bootindex=2 -drive if=none,media=cdrom,id
=drive-sata0-0-1,readonly=on -device ide-cd,bus=ide.1,drive=drive-
sata0-0-1,id=sata0-0-1,bootindex=1 -chardev pty,id=charserial0 -device isa-
serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-
serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-device intel-hda,id=sound0,bus=pci.2,addr=0x1 -device hda-
duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
spicevmc,id=charredir0,name=usbredir -device usb-
redir,chardev=charredir0,id=redir0,bus=usb.0,port=4 -chardev
spicevmc,id=charredir1,name=usbredir -device usb-
redir,chardev=charredir1,id=redir1,bus=usb.0,port=5 -device usb-
host,hostbus=3,hostaddr=5,id=hostdev0,bus=usb.0,port=2 -device usb-
host,hostbus=3,hostaddr=10,id=hostdev1,bus=usb.0,port=3 -device vfio-
pci,host=07:00.0,id=hostdev2,bus=pci.4,addr=0x0,rombar=1 -device vfio-
pci,host=07:00.1,id=hostdev3,bus=pci.5,addr=0x0,rombar=1 -device virtio-
balloon-pci,id=balloon0,bus=pci.6,addr=0x0 -msg timestamp=on
2017-05-23 22:34:33.400+0000: Domain id=3 is tainted: high-privileges
2017-05-23T22:34:33.443075Z qemu-system-x86_64: -chardev pty,id=charserial0:
char device redirected to /dev/pts/2 (label charserial0)
2017-05-23T22:36:32.836901Z qemu-system-x86_64: terminating on signal 15 from
pid 14129 (/usr/bin/libvirtd)
2017-05-23 22:36:34.437+0000: shutting down, reason=destroyed
2017-05-23 23:22:26.933+0000: starting up libvirt version: 3.2.0, qemu
version: 2.9.0, hostname: prototype.filmtek-cloud.intranet
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
QEMU_AUDIO_DRV=none /usr/sbin/qemu-system-x86_64 -name guest=arch-anywhere-nv
,debug-threads=on -S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-arch-
anywhere-nv/master-key.aes -machine pc-q35-2.9,accel=kvm,usb=off,dump-guest-
core=off -cpu Haswell-noTSX,kvm=off -drive
file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on
-drive file=/var/lib/libvirt/qemu/nvram/arch-anywhere-
nv_VARS.fd,if=pflash,format=raw,unit=1 -m 16384 -realtime mlock=off -smp
4,sockets=4,cores=1,threads=1 -uuid 3f1e4dbc-e47f-4dbf-9d21-984efbb2f9dc
-display none -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-arch-anywhere-
nv/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control
-rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet
-no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot
strict=on -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-
bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 -device pcie-root-
port,port=0x8,chassis=3,id=pci.3,bus=pcie.0,multifunction=on,addr=0x1 -device
pcie-root-port,port=0x9,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x1 -device
pcie-root-port,port=0xa,chassis=5,id=pci.5,bus=pcie.0,addr=0x1.0x2 -device
pcie-root-port,port=0xb,chassis=6,id=pci.6,bus=pcie.0,addr=0x1.0x3 -device
pcie-root-port,port=0xc,chassis=7,id=pci.7,bus=pcie.0,addr=0x1.0x4 -device
ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 -device ich9-usb-
uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2
-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -drive
file=/dev/disk/by-id/ata-SanDisk_SDSSDA240G_170735467315,format=raw,if=none,id
=drive-virtio-disk3 -device virtio-blk-pci,scsi=off,bus=pcie.0,addr=0x9,drive
=drive-virtio-disk3,id=virtio-disk3,bootindex=2 -drive if=none,media=cdrom,id
=drive-sata0-0-1,readonly=on -device ide-cd,bus=ide.1,drive=drive-
sata0-0-1,id=sata0-0-1,bootindex=1 -chardev pty,id=charserial0 -device isa-
serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-
serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-device intel-hda,id=sound0,bus=pci.2,addr=0x1 -device hda-
duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
spicevmc,id=charredir0,name=usbredir -device usb-
redir,chardev=charredir0,id=redir0,bus=usb.0,port=4 -chardev
spicevmc,id=charredir1,name=usbredir -device usb-
redir,chardev=charredir1,id=redir1,bus=usb.0,port=5 -device usb-
host,hostbus=3,hostaddr=5,id=hostdev0,bus=usb.0,port=2 -device usb-
host,hostbus=3,hostaddr=10,id=hostdev1,bus=usb.0,port=3 -device vfio-
pci,host=07:00.0,id=hostdev2,bus=pci.4,addr=0x0,rombar=1 -device vfio-
pci,host=07:00.1,id=hostdev3,bus=pci.5,addr=0x0,rombar=1 -device usb-
host,hostbus=3,hostaddr=12,id=hostdev4,bus=usb.0,port=1 -device virtio-
balloon-pci,id=balloon0,bus=pci.6,addr=0x0 -msg timestamp=on
2017-05-23 23:22:26.933+0000: Domain id=4 is tainted: high-privileges
2017-05-23T23:22:26.979538Z qemu-system-x86_64: -chardev pty,id=charserial0:
char device redirected to /dev/pts/2 (label charserial0)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20170525/d835f108/attachment.htm>
More information about the vfio-users
mailing list