[vfio-users] vfio help needed (nvidia GTX 750 Ti persistent code 43)
Francesco Dicarlo
evilsephiroth at gmail.com
Wed Sep 16 14:31:54 UTC 2015
Are you sure your qemu supports Kim hidden tag? Try setting manually hyperv features in features tag off and also hypervclock to off...
Posting ps aux | grep kvm will help us a lot to understand :-)
>>>>> <hyperv>
>>>>> <relaxed state='off'/>
>>>>> <vapic state='off'/>
>>>>> <spinlocks state='off'/>
>>>>> </hyperv>
>>>>> </features>
>>>>>
>>>>> <clock offset='localtime'>
>>>>> <timer name='hypervclock' present='no'/>
>>>>> <timer name='rtc' tickpolicy='catchup'/>
>>>>> <timer name='pit' tickpolicy='delay'/>
>>>>> <timer name='hpet' present='no'/>
>>>>> </clock>
Sent from my iPhone
> On 16 Sep 2015, at 16:10, Mogliii <mogliii at gmx.net> wrote:
>
> Hello,
>
> I have been trying to get vga passthrough working for quite some time
> now. I even ordered a new graphics card just for that purpose.
>
> I explain my setup and configuration below.
>
> My current problem: I can install Windows 8.1 with uefi (gpt partition
> layout) and using virt-manager. Then, I shutdown the VM, remove the
> spice and graphics, and add the pcie MSI GTX 750 Ti Gaming (it has a
> legacy and uefi bios; uefi bios selected by physical switch). When I
> then boot-up the VM, I see the OVMF bios on the physical screen attached
> to the GTX750 Ti (sometimes I get graphics glitches i.e. all white) and
> Windows 8.1 boots into 800x600. I see that a Nvidia GTX750Ti is
> recognized in the device manager. After installing driver (I tried
> 335.23, 340.52, 341.44 and most recent) the gpu is shown with yellow
> exclamation mark with error code 43 and I have to restart. After
> restart, I see again the bios and the boot splash, but then the screen
> goes into power save mode because it does not get any signal just when
> the login screen is supposed to appear.
> I can login via vnc and see that the gpu has error code 43.
> De-installing the driver and restarting I will go back to a 800x600
> desktop on my physical screen.
>
> Any help with my setup would be immensely appreciated. I have another
> spare ASUS HD6570, but it does not have uefi bios. I am not sure if I
> would need i915 patch in that case? Does that patch only apply to Intel
> CPUs?
>
> Is the 00:02.2 pci device the problem?
>
> I would really appreciate any hint as I currently exhausted all options
> I see and I would like to avoid spending more money on video cards that
> I'm not sure I can even use.
>
> ==========================
> SETUP
> =========
>
> [[[ 1. Physical setup
> Asus A88x-PRO
> AMD A10-7850K (on-board GPU used for linux host with radeon driver
> (fglrx not installed))
> 12 GB memory
> MSI GTX 750 Ti Gaming (bios can be switched between legacy/uefi by
> hardware switch; using uefi)
>
> GPU is installed in middle pcie 16x slot (black, not golden) due to
> iommu grouping (see below)
>
>
> [[[ 2. OS
> UBuntu 15.04 64 bit, unpatched
> $ uname -a
> Linux host 3.19.0-28-generic #30-Ubuntu SMP Mon Aug 31 15:52:51 UTC 2015
> x86_64 x86_64 x86_64 GNU/Linux
>
>
> [[[ 3. Relevant packages (not sure if these are all)
> virt-manager 1:1.2.1-1~getdeb1 (get deb ppa)
> qemu 1:2.2+dfsg-5expubuntu9.4
> libvirt-bin 1.2.12-0ubuntu14.2
> qemu-kvm 1:2.2+dfsg-5expubuntu9.4
>
> [[[ 4. Bios settings
> Set to boot from on board GPU. AMD-V and IOMMU activated. Legacy boot
> for ubuntu host
>
> [[[ 5. iommu grouping (iommu.sh output)
> ### Group 2 ###
> 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1424
> 00:02.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1425
> 01:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce
> GTX 750 Ti] (rev a2)
> 01:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)
>
> [[[ 6. pci identifier
> 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107
> [GeForce GTX 750 Ti] [10de:1380] (rev a2)
> Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3102]
> Kernel driver in use: pci-stub
> 01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fbc] (rev a1)
> Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3102]
> Kernel driver in use: pci-stub
>
> [[[ 7. /proc/cmdline
> BOOT_IMAGE=/vmlinuz-3.19.0-28-generic root=/dev/mapper/main-root ro
> rootflags=subvol=@ quiet splash nomdmonddf nomdmonisw amd_iommu=on
> iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1
> pci-stub.ids=10de:0fbc,10de:1380 vt.handoff=7
>
> [[[ 8. /etc/initramfs-tools/modules
> pci_stub ids=10de:0fbc,10de:1380
>
> [[[ 9. /etc/modules
> lp
> rtc
> pci_stub
> vfio
> vfio_iommu_type1
> vfio_pci
> kvm
> kvm_amd
>
> [[[ 10. # dmseg | grep pci-stub
> [ 1.263269] pci-stub: add 10DE:0FBC sub=FFFFFFFF:FFFFFFFF
> cls=00000000/00000000
> [ 1.263287] pci-stub 0000:01:00.1: claimed by stub
> [ 1.263296] pci-stub: add 10DE:1380 sub=FFFFFFFF:FFFFFFFF
> cls=00000000/00000000
> [ 1.263304] pci-stub 0000:01:00.0: claimed by stub
>
> [[[ 11. /usr/windows.sh (executed before running vm. Afterwards lspci
> -nnk shows "vfio-pci" as kernel driver in use for 01:00.0 and 01:00.1)
> #!/bin/bash
>
> configfile=/etc/vfio_pci1.cfg
> vfiobind() {
> dev="$1"
> vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
> device=$(cat /sys/bus/pci/devices/$dev/device)
> if [ -e /sys/bus/pci/devices/$dev/driver ]; then
> echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
> fi
> echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
> }
> modprobe vfio-pci
> cat $configfile | while read line;do
> echo $line | grep ^# >/dev/null 2>&1 && continue
> vfiobind $line
> done
> exit 0
>
> [[[ 12. /etc/vfio_pci1.cfg
> 0000:01:00.0
> 0000:01:00.1
>
> [[[ 13. OVMF-bios
> Got from sourceforge
> (edk2.git-ovmf-x64-0-20150910.b1206.g5abc2a7.noarch.rpm) and extracted.
> Then, made individual copy of OVMF-pure-efi.fd (md5
> fbb13f2d4be2f9d72472628f885f5cb4 ) for VM.
>
> [[[ 14. Created VM in virt-manager using the following configuration
> (configuration shown after installation, i.e. no cdroms, but before
> attaching pci)
> <domain type='kvm'>
> <name>win81</name>
> <uuid>redacted</uuid>
> <memory unit='KiB'>4194304</memory>
> <currentMemory unit='KiB'>4194304</currentMemory>
> <vcpu placement='static'>2</vcpu>
> <os>
> <type arch='x86_64' machine='pc-i440fx-utopic'>hvm</type>
> <loader type='rom'>/usr/share/ovmf/OVMF-pure-efi_win81.fd</loader>
> <boot dev='hd'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> <pae/>
> <kvm>
> <hidden state='on'/>
> </kvm>
> </features>
> <cpu mode='host-passthrough'>
> </cpu>
> <clock offset='localtime'>
> <timer name='rtc' tickpolicy='catchup'/>
> <timer name='pit' tickpolicy='delay'/>
> </clock>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>restart</on_crash>
> <pm>
> <suspend-to-mem enabled='no'/>
> <suspend-to-disk enabled='no'/>
> </pm>
> <devices>
> <emulator>/usr/bin/qemu-system-x86_64</emulator>
> <disk type='block' device='disk'>
> <driver name='qemu' type='raw' cache='none' io='native'/>
> <source dev='/dev/main/Win7xen'/>
> <target dev='vda' bus='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
> </disk>
> <controller type='usb' index='0' model='ich9-ehci1'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x7'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci1'>
> <master startport='0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0' multifunction='on'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci2'>
> <master startport='2'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x1'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci3'>
> <master startport='4'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x2'/>
> </controller>
> <controller type='pci' index='0' model='pci-root'/>
> <controller type='virtio-serial' index='0'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
> </controller>
> <interface type='bridge'>
> <mac address='52:54:00:ab:b6:10'/>
> <source bridge='br1'/>
> <model type='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </interface>
> <input type='mouse' bus='ps2'/>
> <input type='keyboard' bus='ps2'/>
> <graphics type='spice' autoport='yes'/>
> <video>
> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
> </video>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x0603'/>
> <product id='0x1605'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x05ac'/>
> <product id='0x1006'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x05ac'/>
> <product id='0x024f'/>
> </source>
> </hostdev>
> <memballoon model='virtio'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
> </memballoon>
> </devices>
> </domain>
>
> [[[ 15. After installation and installation of updates, shut down the VM
>
> [[[ 16. Attached PCI devices and removed video and display spice. Below
> is virsh configuration after attaching pcie passthrough.
> <domain type='kvm'>
> <name>win81</name>
> <uuid>redacted</uuid>
> <memory unit='KiB'>4194304</memory>
> <currentMemory unit='KiB'>4194304</currentMemory>
> <vcpu placement='static'>2</vcpu>
> <os>
> <type arch='x86_64' machine='pc-i440fx-utopic'>hvm</type>
> <loader type='rom'>/usr/share/ovmf/OVMF-pure-efi_win81.fd</loader>
> <boot dev='hd'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> <pae/>
> <kvm>
> <hidden state='on'/>
> </kvm>
> </features>
> <cpu mode='host-passthrough'>
> </cpu>
> </clock>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>restart</on_crash>
> <pm>
> <suspend-to-mem enabled='no'/>
> <suspend-to-disk enabled='no'/>
> </pm>
> <devices>
> <emulator>/usr/bin/qemu-system-x86_64</emulator>
> <disk type='block' device='disk'>
> <driver name='qemu' type='raw' cache='none' io='native'/>
> <source dev='/dev/main/Win7xen'/>
> <target dev='vda' bus='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
> </disk>
> <controller type='usb' index='0' model='ich9-ehci1'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x7'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci1'>
> <master startport='0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0' multifunction='on'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci2'>
> <master startport='2'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x1'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci3'>
> <master startport='4'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x2'/>
> </controller>
> <controller type='pci' index='0' model='pci-root'/>
> <controller type='virtio-serial' index='0'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
> </controller>
> <interface type='bridge'>
> <mac address='52:54:00:ab:b6:10'/>
> <source bridge='br1'/>
> <model type='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </interface>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x0603'/>
> <product id='0x1605'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x05ac'/>
> <product id='0x1006'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x05ac'/>
> <product id='0x024f'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
> </hostdev>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
> function='0x0'/>
> </hostdev>
> <memballoon model='virtio'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
> </memballoon>
> </devices>
> </domain>
>
> [[[ 17. What else did I try
> - Different nvidia drivers (see top of email)
> - Attaching 00:02.2 as well, but cannot do vfio-pci assignement so failed
> # virsh nodedev-list --tree
> +- pci_0000_00_02_0
> +- pci_0000_00_02_2
> | |
> | +- pci_0000_01_00_0
> | +- pci_0000_01_00_1
> |
> +- pci_0000_00_03_0
>
> # lspci -nnk (extract)
> 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device
> [1022:1424]
> 00:02.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device
> [1022:1425]
> Kernel driver in use: pcieport
>
> - Reinstall the whole VM with seabios and using the GTX 750 Ti with
> legacy bios (physical switch on card). But no picture was shown on
> screen after attaching, not even the bios.
> - Tried to use q35 emulation, but not successful. It does not work with
> ovmf? I.e. I cannot use the qemu:cmd or ioh3420.
>
>
>
>
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150916/23deb0e8/attachment.htm>
More information about the vfio-users
mailing list