[vfio-users] vfio help needed (nvidia GTX 750 Ti persistent code 43)
Blank Field
ihatethisfield at gmail.com
Wed Sep 16 14:36:25 UTC 2015
Oh, and remove that "allow_unsafe_interrupts" snippet, we're telling people
not to add it if they weren't told so in dmesg for two or more years but
still.
On Sep 16, 2015 5:32 PM, "Blank Field" <ihatethisfield at gmail.com> wrote:
> You've gone the right way in running OVMF instead of VGA, as it is broken
> on your platform.
> Code 43: nvidia drivers have a check code that disables the card if it
> appears to be attached to a VM. Do you have hyper-v enlightenments turned
> off and kvm cpuid signature also turned off?
> Otherwise code 43 is just as bad as code 10: the driver fails to start the
> device for whatever reason it has.
> On Sep 16, 2015 5:11 PM, "Mogliii" <mogliii at gmx.net> wrote:
>
>> Hello,
>>
>> I have been trying to get vga passthrough working for quite some time
>> now. I even ordered a new graphics card just for that purpose.
>>
>> I explain my setup and configuration below.
>>
>> My current problem: I can install Windows 8.1 with uefi (gpt partition
>> layout) and using virt-manager. Then, I shutdown the VM, remove the
>> spice and graphics, and add the pcie MSI GTX 750 Ti Gaming (it has a
>> legacy and uefi bios; uefi bios selected by physical switch). When I
>> then boot-up the VM, I see the OVMF bios on the physical screen attached
>> to the GTX750 Ti (sometimes I get graphics glitches i.e. all white) and
>> Windows 8.1 boots into 800x600. I see that a Nvidia GTX750Ti is
>> recognized in the device manager. After installing driver (I tried
>> 335.23, 340.52, 341.44 and most recent) the gpu is shown with yellow
>> exclamation mark with error code 43 and I have to restart. After
>> restart, I see again the bios and the boot splash, but then the screen
>> goes into power save mode because it does not get any signal just when
>> the login screen is supposed to appear.
>> I can login via vnc and see that the gpu has error code 43.
>> De-installing the driver and restarting I will go back to a 800x600
>> desktop on my physical screen.
>>
>> Any help with my setup would be immensely appreciated. I have another
>> spare ASUS HD6570, but it does not have uefi bios. I am not sure if I
>> would need i915 patch in that case? Does that patch only apply to Intel
>> CPUs?
>>
>> Is the 00:02.2 pci device the problem?
>>
>> I would really appreciate any hint as I currently exhausted all options
>> I see and I would like to avoid spending more money on video cards that
>> I'm not sure I can even use.
>>
>> ==========================
>> SETUP
>> =========
>>
>> [[[ 1. Physical setup
>> Asus A88x-PRO
>> AMD A10-7850K (on-board GPU used for linux host with radeon driver
>> (fglrx not installed))
>> 12 GB memory
>> MSI GTX 750 Ti Gaming (bios can be switched between legacy/uefi by
>> hardware switch; using uefi)
>>
>> GPU is installed in middle pcie 16x slot (black, not golden) due to
>> iommu grouping (see below)
>>
>>
>> [[[ 2. OS
>> UBuntu 15.04 64 bit, unpatched
>> $ uname -a
>> Linux host 3.19.0-28-generic #30-Ubuntu SMP Mon Aug 31 15:52:51 UTC 2015
>> x86_64 x86_64 x86_64 GNU/Linux
>>
>>
>> [[[ 3. Relevant packages (not sure if these are all)
>> virt-manager 1:1.2.1-1~getdeb1 (get deb ppa)
>> qemu 1:2.2+dfsg-5expubuntu9.4
>> libvirt-bin 1.2.12-0ubuntu14.2
>> qemu-kvm 1:2.2+dfsg-5expubuntu9.4
>>
>> [[[ 4. Bios settings
>> Set to boot from on board GPU. AMD-V and IOMMU activated. Legacy boot
>> for ubuntu host
>>
>> [[[ 5. iommu grouping (iommu.sh output)
>> ### Group 2 ###
>> 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 1424
>> 00:02.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 1425
>> 01:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce
>> GTX 750 Ti] (rev a2)
>> 01:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)
>>
>> [[[ 6. pci identifier
>> 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107
>> [GeForce GTX 750 Ti] [10de:1380] (rev a2)
>> Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3102]
>> Kernel driver in use: pci-stub
>> 01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fbc] (rev
>> a1)
>> Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3102]
>> Kernel driver in use: pci-stub
>>
>> [[[ 7. /proc/cmdline
>> BOOT_IMAGE=/vmlinuz-3.19.0-28-generic root=/dev/mapper/main-root ro
>> rootflags=subvol=@ quiet splash nomdmonddf nomdmonisw amd_iommu=on
>> iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1
>> pci-stub.ids=10de:0fbc,10de:1380 vt.handoff=7
>>
>> [[[ 8. /etc/initramfs-tools/modules
>> pci_stub ids=10de:0fbc,10de:1380
>>
>> [[[ 9. /etc/modules
>> lp
>> rtc
>> pci_stub
>> vfio
>> vfio_iommu_type1
>> vfio_pci
>> kvm
>> kvm_amd
>>
>> [[[ 10. # dmseg | grep pci-stub
>> [ 1.263269] pci-stub: add 10DE:0FBC sub=FFFFFFFF:FFFFFFFF
>> cls=00000000/00000000
>> [ 1.263287] pci-stub 0000:01:00.1: claimed by stub
>> [ 1.263296] pci-stub: add 10DE:1380 sub=FFFFFFFF:FFFFFFFF
>> cls=00000000/00000000
>> [ 1.263304] pci-stub 0000:01:00.0: claimed by stub
>>
>> [[[ 11. /usr/windows.sh (executed before running vm. Afterwards lspci
>> -nnk shows "vfio-pci" as kernel driver in use for 01:00.0 and 01:00.1)
>> #!/bin/bash
>>
>> configfile=/etc/vfio_pci1.cfg
>> vfiobind() {
>> dev="$1"
>> vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
>> device=$(cat /sys/bus/pci/devices/$dev/device)
>> if [ -e /sys/bus/pci/devices/$dev/driver ]; then
>> echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
>> fi
>> echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
>> }
>> modprobe vfio-pci
>> cat $configfile | while read line;do
>> echo $line | grep ^# >/dev/null 2>&1 && continue
>> vfiobind $line
>> done
>> exit 0
>>
>> [[[ 12. /etc/vfio_pci1.cfg
>> 0000:01:00.0
>> 0000:01:00.1
>>
>> [[[ 13. OVMF-bios
>> Got from sourceforge
>> (edk2.git-ovmf-x64-0-20150910.b1206.g5abc2a7.noarch.rpm) and extracted.
>> Then, made individual copy of OVMF-pure-efi.fd (md5
>> fbb13f2d4be2f9d72472628f885f5cb4 ) for VM.
>>
>> [[[ 14. Created VM in virt-manager using the following configuration
>> (configuration shown after installation, i.e. no cdroms, but before
>> attaching pci)
>> <domain type='kvm'>
>> <name>win81</name>
>> <uuid>redacted</uuid>
>> <memory unit='KiB'>4194304</memory>
>> <currentMemory unit='KiB'>4194304</currentMemory>
>> <vcpu placement='static'>2</vcpu>
>> <os>
>> <type arch='x86_64' machine='pc-i440fx-utopic'>hvm</type>
>> <loader type='rom'>/usr/share/ovmf/OVMF-pure-efi_win81.fd</loader>
>> <boot dev='hd'/>
>> </os>
>> <features>
>> <acpi/>
>> <apic/>
>> <pae/>
>> <kvm>
>> <hidden state='on'/>
>> </kvm>
>> </features>
>> <cpu mode='host-passthrough'>
>> </cpu>
>> <clock offset='localtime'>
>> <timer name='rtc' tickpolicy='catchup'/>
>> <timer name='pit' tickpolicy='delay'/>
>> </clock>
>> <on_poweroff>destroy</on_poweroff>
>> <on_reboot>restart</on_reboot>
>> <on_crash>restart</on_crash>
>> <pm>
>> <suspend-to-mem enabled='no'/>
>> <suspend-to-disk enabled='no'/>
>> </pm>
>> <devices>
>> <emulator>/usr/bin/qemu-system-x86_64</emulator>
>> <disk type='block' device='disk'>
>> <driver name='qemu' type='raw' cache='none' io='native'/>
>> <source dev='/dev/main/Win7xen'/>
>> <target dev='vda' bus='virtio'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>> function='0x0'/>
>> </disk>
>> <controller type='usb' index='0' model='ich9-ehci1'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x7'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci1'>
>> <master startport='0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x0' multifunction='on'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci2'>
>> <master startport='2'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x1'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci3'>
>> <master startport='4'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x2'/>
>> </controller>
>> <controller type='pci' index='0' model='pci-root'/>
>> <controller type='virtio-serial' index='0'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
>> function='0x0'/>
>> </controller>
>> <interface type='bridge'>
>> <mac address='52:54:00:ab:b6:10'/>
>> <source bridge='br1'/>
>> <model type='virtio'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
>> function='0x0'/>
>> </interface>
>> <input type='mouse' bus='ps2'/>
>> <input type='keyboard' bus='ps2'/>
>> <graphics type='spice' autoport='yes'/>
>> <video>
>> <model type='qxl' ram='65536' vram='65536' vgamem='16384'
>> heads='1'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
>> function='0x0'/>
>> </video>
>> <hostdev mode='subsystem' type='usb' managed='yes'>
>> <source>
>> <vendor id='0x0603'/>
>> <product id='0x1605'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='yes'>
>> <source>
>> <vendor id='0x05ac'/>
>> <product id='0x1006'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='yes'>
>> <source>
>> <vendor id='0x05ac'/>
>> <product id='0x024f'/>
>> </source>
>> </hostdev>
>> <memballoon model='virtio'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
>> function='0x0'/>
>> </memballoon>
>> </devices>
>> </domain>
>>
>> [[[ 15. After installation and installation of updates, shut down the VM
>>
>> [[[ 16. Attached PCI devices and removed video and display spice. Below
>> is virsh configuration after attaching pcie passthrough.
>> <domain type='kvm'>
>> <name>win81</name>
>> <uuid>redacted</uuid>
>> <memory unit='KiB'>4194304</memory>
>> <currentMemory unit='KiB'>4194304</currentMemory>
>> <vcpu placement='static'>2</vcpu>
>> <os>
>> <type arch='x86_64' machine='pc-i440fx-utopic'>hvm</type>
>> <loader type='rom'>/usr/share/ovmf/OVMF-pure-efi_win81.fd</loader>
>> <boot dev='hd'/>
>> </os>
>> <features>
>> <acpi/>
>> <apic/>
>> <pae/>
>> <kvm>
>> <hidden state='on'/>
>> </kvm>
>> </features>
>> <cpu mode='host-passthrough'>
>> </cpu>
>> </clock>
>> <on_poweroff>destroy</on_poweroff>
>> <on_reboot>restart</on_reboot>
>> <on_crash>restart</on_crash>
>> <pm>
>> <suspend-to-mem enabled='no'/>
>> <suspend-to-disk enabled='no'/>
>> </pm>
>> <devices>
>> <emulator>/usr/bin/qemu-system-x86_64</emulator>
>> <disk type='block' device='disk'>
>> <driver name='qemu' type='raw' cache='none' io='native'/>
>> <source dev='/dev/main/Win7xen'/>
>> <target dev='vda' bus='virtio'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>> function='0x0'/>
>> </disk>
>> <controller type='usb' index='0' model='ich9-ehci1'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x7'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci1'>
>> <master startport='0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x0' multifunction='on'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci2'>
>> <master startport='2'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x1'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci3'>
>> <master startport='4'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
>> function='0x2'/>
>> </controller>
>> <controller type='pci' index='0' model='pci-root'/>
>> <controller type='virtio-serial' index='0'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
>> function='0x0'/>
>> </controller>
>> <interface type='bridge'>
>> <mac address='52:54:00:ab:b6:10'/>
>> <source bridge='br1'/>
>> <model type='virtio'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
>> function='0x0'/>
>> </interface>
>> <hostdev mode='subsystem' type='usb' managed='yes'>
>> <source>
>> <vendor id='0x0603'/>
>> <product id='0x1605'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='yes'>
>> <source>
>> <vendor id='0x05ac'/>
>> <product id='0x1006'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='yes'>
>> <source>
>> <vendor id='0x05ac'/>
>> <product id='0x024f'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='pci' managed='yes'>
>> <source>
>> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
>> </source>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
>> function='0x0'/>
>> </hostdev>
>> <hostdev mode='subsystem' type='pci' managed='yes'>
>> <source>
>> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
>> </source>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
>> function='0x0'/>
>> </hostdev>
>> <memballoon model='virtio'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
>> function='0x0'/>
>> </memballoon>
>> </devices>
>> </domain>
>>
>> [[[ 17. What else did I try
>> - Different nvidia drivers (see top of email)
>> - Attaching 00:02.2 as well, but cannot do vfio-pci assignement so failed
>> # virsh nodedev-list --tree
>> +- pci_0000_00_02_0
>> +- pci_0000_00_02_2
>> | |
>> | +- pci_0000_01_00_0
>> | +- pci_0000_01_00_1
>> |
>> +- pci_0000_00_03_0
>>
>> # lspci -nnk (extract)
>> 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device
>> [1022:1424]
>> 00:02.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device
>> [1022:1425]
>> Kernel driver in use: pcieport
>>
>> - Reinstall the whole VM with seabios and using the GTX 750 Ti with
>> legacy bios (physical switch on card). But no picture was shown on
>> screen after attaching, not even the bios.
>> - Tried to use q35 emulation, but not successful. It does not work with
>> ovmf? I.e. I cannot use the qemu:cmd or ioh3420.
>>
>>
>>
>>
>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150916/b1380319/attachment.htm>
More information about the vfio-users
mailing list