[vfio-users] KVM Nvidia Passtrough Code 43 and freezes

Marco Stagge spiritblackdiamont at gmail.com
Fri Aug 28 18:11:46 UTC 2015


Uhm yeah, i dont know how i did this but i think i was the initramfs......

i checked lspci -k on my installation, and the 750ti wasnt assigned to 
pci-stub.....after checking the initramfs-tools/modules, i tried to 
update the initramfs(i thought i did forget this) and debian throw me a 
strange error.... after googlin around and solving this problem, i 
rebootet my system and reinstalled my win10 vm.

The devicemanager display no error on the 750ti and i'm on fullhd atm :)
The nvidia panel runs without problems, driver is 353.54 :)

Well thank you all, guys :)
I'll do a benchmark etc now, yay ^_^

greetings
Marco

Am 28.08.2015 um 19:31 schrieb Jon Panozzo:
> Hmm.  Can you ssh into the system and type lspci -k and copy/paste the 
> results back here?  This is very odd behavior indeed.
>
> One other thing we can try too is to pci-stub this device and reboot. 
>  Not sure how you were doing it on your system, but with unRAID, you 
> do this by logging in via ssh, then running the command lspci -n. 
>  Locate the row for your device (the 02:00.0 and .1), then copy the 
> vendor/product ID.
>
> Back on the webGui, go to the Main tab, then click on the word “Flash” 
> under the devices column.  Now go to the syslinux section and modify 
> the /append line.  Here’s mine, where I stub numerous devices:
>
> default /syslinux/menu.c32
> menu title Lime Technology
> prompt 0
> timeout 50
> label unRAID OS
>   menu default
>   kernel /bzimage
>   append 
> pci-stub.ids=1002:68f9,1002:aa68,10de:1004,10de:0e1a,1a0a:6202,8086:0412 
> pcie_acs_override=downstream initrd=/bzroot
> label unRAID OS Safe Mode (no plugins)
>   kernel /bzimage
>   append initrd=/bzroot unraidsafemode
> label Memtest86+
>   kernel /memtest
> label Xen/unRAID OS
>   kernel /syslinux/mboot.c32
>   append /xen dom0_mem=4096M iommu=on --- /bzimage 
> xen-pciback.hide=(01:00.0)(01:00.1)(02:00.0)(02:00.1) intel_iommu=on 
> --- /bzroot
> label Xen/unRAID OS Safe Mode (no plugins)
>   kernel /syslinux/mboot.c32
>   append /xen --- /bzimage --- /bzroot unraidsafemode
>
> You shouldn’t need to stub the devices in unRAID because we don’t load 
> GPU drivers, so nothing should be bound to the device.
>
> If neither of these work, you may have me stumped :-(
>
> As a last resort, you can go to Tools -> Diagnostics and click 
> collect, then send me that zip file for further analysis.
>
> - Jon
>
>> On Aug 28, 2015, at 12:13 PM, Marco Stagge 
>> <spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>> 
>> wrote:
>>
>> okay here:
>>
>> /sys/kernel/iommu_groups/0/devices/0000:00:00.0
>> /sys/kernel/iommu_groups/1/devices/0000:00:02.0
>> /sys/kernel/iommu_groups/2/devices/0000:00:03.0
>> /sys/kernel/iommu_groups/3/devices/0000:00:04.0
>> /sys/kernel/iommu_groups/4/devices/0000:00:05.0
>> /sys/kernel/iommu_groups/5/devices/0000:00:06.0
>> /sys/kernel/iommu_groups/6/devices/0000:00:07.0
>> /sys/kernel/iommu_groups/7/devices/0000:00:09.0
>> /sys/kernel/iommu_groups/8/devices/0000:00:0a.0
>> /sys/kernel/iommu_groups/9/devices/0000:00:11.0
>> /sys/kernel/iommu_groups/10/devices/0000:00:12.0
>> /sys/kernel/iommu_groups/10/devices/0000:00:12.2
>> /sys/kernel/iommu_groups/11/devices/0000:00:13.0
>> /sys/kernel/iommu_groups/11/devices/0000:00:13.2
>> /sys/kernel/iommu_groups/12/devices/0000:00:14.0
>> /sys/kernel/iommu_groups/13/devices/0000:00:14.2
>> /sys/kernel/iommu_groups/14/devices/0000:00:14.3
>> /sys/kernel/iommu_groups/15/devices/0000:00:14.4
>> /sys/kernel/iommu_groups/15/devices/0000:09:06.0
>> /sys/kernel/iommu_groups/16/devices/0000:00:14.5
>> /sys/kernel/iommu_groups/17/devices/0000:00:15.0
>> /sys/kernel/iommu_groups/18/devices/0000:00:16.0
>> /sys/kernel/iommu_groups/18/devices/0000:00:16.2
>> /sys/kernel/iommu_groups/19/devices/0000:01:00.0
>> /sys/kernel/iommu_groups/19/devices/0000:01:00.1
>> /sys/kernel/iommu_groups/20/devices/0000:02:00.0
>> /sys/kernel/iommu_groups/20/devices/0000:02:00.1
>> /sys/kernel/iommu_groups/21/devices/0000:03:00.0
>> /sys/kernel/iommu_groups/22/devices/0000:04:00.0
>> /sys/kernel/iommu_groups/23/devices/0000:05:00.0
>> /sys/kernel/iommu_groups/24/devices/0000:06:00.0
>> /sys/kernel/iommu_groups/25/devices/0000:07:00.0
>> /sys/kernel/iommu_groups/26/devices/0000:08:00.0
>> /sys/kernel/iommu_groups/27/devices/0000:0a:00.0
>> /sys/kernel/iommu_groups/27/devices/0000:0a:00.1
>>
>>
>> Am 28.08.2015 um 19:11 schrieb Jon Panozzo:
>>> Under Tools -> System Devices, please copy and paste the IOMMU 
>>> groups section back in an e-mail to me.
>>>
>>> - Jon
>>>
>>>> On Aug 28, 2015, at 12:09 PM, Marco Stagge 
>>>> <spiritblackdiamont at gmail.com 
>>>> <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>
>>>> done, but still the same error :(
>>>>
>>>> Am 28.08.2015 um 18:59 schrieb Jon Panozzo:
>>>>> Ok, that is because you need to enable PCI ACS Override.  Under 
>>>>> Settings -> VM Manager, enable PCI ACS Override.  Then reboot your 
>>>>> system.
>>>>>
>>>>>> On Aug 28, 2015, at 11:57 AM, Marco Stagge 
>>>>>> <spiritblackdiamont at gmail.com 
>>>>>> <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>>>
>>>>>> Hi Jon,
>>>>>>
>>>>>> i've tried to create a vm but its not possible:
>>>>>> Error creating VM: internal error: early end of file from 
>>>>>> monitor: possible problem:
>>>>>> 2015-08-28T16:54:50.210851Z qemu-system-x86_64: -device 
>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: 
>>>>>> vfio: failed to set iommu for container: Operation not permitted
>>>>>> 2015-08-28T16:54:50.210877Z qemu-system-x86_64: -device 
>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: 
>>>>>> vfio: failed to setup container for group 20
>>>>>> 2015-08-28T16:54:50.210886Z qemu-system-x86_64: -device 
>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: 
>>>>>> vfio: failed to get group 20
>>>>>> 2015-08-28T16:54:50.210898Z qemu-system-x86_64: -device 
>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: 
>>>>>> Device initialization failed
>>>>>> 2015-08-28T16:54:50.210910Z qemu-system-x86_64: -device 
>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: 
>>>>>> Device 'vfio-pci' could not be initialized
>>>>>>
>>>>>> did i missed something?
>>>>>>
>>>>>> greetings
>>>>>> Marco
>>>>>>
>>>>>> Am 28.08.2015 um 18:05 schrieb Jon Panozzo:
>>>>>>> Hi Marco,
>>>>>>>
>>>>>>> Not sure if you want to go down this road or not, but a 
>>>>>>> relatively quick path to checking to see if this is a kernel/OS 
>>>>>>> config issue would be to load up our OS (unRAID) which boots of 
>>>>>>> a USB stick and see if you have success there.  We’ve tuned our 
>>>>>>> OS for use with VFIO/VGA assignment, and we don’t load a desktop 
>>>>>>> GUI at all (nor any graphics devices).  The OS uses libvirt and 
>>>>>>> provides a webGui for testing various configs (OVMF vs. SeaBIOS 
>>>>>>> and i440fx vs. Q35).  Hopefully this isn’t seen as a shameless 
>>>>>>> plug on our OS here, but it could be a way to narrow down the 
>>>>>>> issue, and it’s free to try (http://lime-technology.com).
>>>>>>>
>>>>>>> I can tell you that the NVIDIA drivers have nothing to do with 
>>>>>>> it.  I have multiple NVIDIA GPUs that are running on multiple 
>>>>>>> systems.  So long as you don’t enable the hyper-v enlightenments 
>>>>>>> through QEMU/KVM and specify kvm=off like you are, the card does 
>>>>>>> not detect that it is running in a virtual machine / render the 
>>>>>>> code 43 error.  There are other issues that can cause code 43 
>>>>>>> unrelated to detection by the device drivers, but narrowing down 
>>>>>>> the specifics can be a challenge.
>>>>>>>
>>>>>>> - Jon
>>>>>>>
>>>>>>>> On Aug 28, 2015, at 10:58 AM, Marco Stagge 
>>>>>>>> <spiritblackdiamont at gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi Jonathan,
>>>>>>>>
>>>>>>>> this is a AMD Board without a integrated GPU :)
>>>>>>>>
>>>>>>>> Yep, nouveau driver don't load on this card but the nouveau 
>>>>>>>> driver tells me on boot some strange lines at dmesg but can 
>>>>>>>> display the EFI Screen etc.
>>>>>>>>
>>>>>>>> Yeah i saw on the bbs from archlinux some guys with nvidia 
>>>>>>>> cards running, but this thread is a bit "old". So i think 
>>>>>>>> nvidia changed the driver to recognize the machine as 
>>>>>>>> virtual(Windows 10 does).
>>>>>>>>
>>>>>>>> Without the romfile, it wont display anything(same on the 
>>>>>>>> radeon cards) :o
>>>>>>>>
>>>>>>>> Here's my lspci -k:
>>>>>>>> 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] RD890 
>>>>>>>> PCI to PCI bridge (external gfx0 port B)
>>>>>>>> 00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O 
>>>>>>>> Memory Management Unit (IOMMU)
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] RD990 
>>>>>>>> I/O Memory Management Unit (IOMMU)
>>>>>>>> 00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port B)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port C)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port D)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port E)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port F)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port G)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port H)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> RD890 PCI to PCI bridge (external gfx1 port A)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. Device 84dd
>>>>>>>>        Kernel driver in use: ahci
>>>>>>>> 00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>        Kernel driver in use: ohci-pci
>>>>>>>> 00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>        Kernel driver in use: ehci-pci
>>>>>>>> 00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 
>>>>>>>> SMBus Controller (rev 42)
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 
>>>>>>>> SMBus Controller
>>>>>>>>        Kernel driver in use: piix4_smbus
>>>>>>>> 00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SBx00 Azalia (Intel HDA) (rev 40)
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. M4A89GTD PRO/USB3 
>>>>>>>> Motherboard
>>>>>>>> 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 LPC host controller
>>>>>>>> 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SBx00 PCI to PCI Bridge (rev 40)
>>>>>>>> 00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
>>>>>>>>        Kernel driver in use: ohci-pci
>>>>>>>> 00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
>>>>>>>>        Kernel driver in use: pcieport
>>>>>>>> 00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 
>>>>>>>> 15h Processor Function 0
>>>>>>>> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 
>>>>>>>> 15h Processor Function 1
>>>>>>>> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 
>>>>>>>> 15h Processor Function 2
>>>>>>>> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 
>>>>>>>> 15h Processor Function 3
>>>>>>>>        Kernel driver in use: k10temp
>>>>>>>> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 
>>>>>>>> 15h Processor Function 4
>>>>>>>>        Kernel driver in use: fam15h_power
>>>>>>>> 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 
>>>>>>>> 15h Processor Function 5
>>>>>>>> 01:00.0 VGA compatible controller: NVIDIA Corporation GF119 
>>>>>>>> [GeForce GT 520] (rev a1)
>>>>>>>>        Subsystem: ZOTAC International (MCO) Ltd. Device 3214
>>>>>>>>        Kernel driver in use: nouveau
>>>>>>>> 01:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio 
>>>>>>>> Controller (rev a1)
>>>>>>>>        Subsystem: ZOTAC International (MCO) Ltd. Device 3214
>>>>>>>> 02:00.0 VGA compatible controller: NVIDIA Corporation GM107 
>>>>>>>> [GeForce GTX 750 Ti] (rev a2)
>>>>>>>>        Subsystem: eVga.com <http://evga.com/>. Corp. Device 3753
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 02:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)
>>>>>>>>        Subsystem: eVga.com <http://evga.com/>. Corp. Device 3753
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. 
>>>>>>>> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. P8P67 and other 
>>>>>>>> motherboards
>>>>>>>>        Kernel driver in use: r8169
>>>>>>>> 04:00.0 USB controller: ASMedia Technology Inc. ASM1042 
>>>>>>>> SuperSpeed USB Host Controller
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 05:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA 
>>>>>>>> Controller (rev 10)
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard
>>>>>>>>        Kernel driver in use: ahci
>>>>>>>> 06:00.0 USB controller: ASMedia Technology Inc. ASM1042 
>>>>>>>> SuperSpeed USB Host Controller
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. 
>>>>>>>> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
>>>>>>>>        Subsystem: Device 7470:3468
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 08:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA 
>>>>>>>> Controller (rev 10)
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard
>>>>>>>>        Kernel driver in use: ahci
>>>>>>>> 09:06.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 
>>>>>>>> [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)
>>>>>>>>        Subsystem: ASUSTeK Computer Inc. Motherboard
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 0a:00.0 VGA compatible controller: Advanced Micro Devices, Inc. 
>>>>>>>> [AMD/ATI] Turks XT [Radeon HD 6670/7670]
>>>>>>>>        Subsystem: PC Partner Limited / Sapphire Technology 
>>>>>>>> Device e195
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>> 0a:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] 
>>>>>>>> Turks/Whistler HDMI Audio [Radeon HD 6000 Series]
>>>>>>>>        Subsystem: PC Partner Limited / Sapphire Technology 
>>>>>>>> Device aa90
>>>>>>>>        Kernel driver in use: vfio-pci
>>>>>>>>
>>>>>>>> greetings
>>>>>>>> Marco - Spirit
>>>>>>>>
>>>>>>>> Am 28.08.2015 um 17:49 schrieb Jon Panozzo:
>>>>>>>>> Hi Marco,
>>>>>>>>>
>>>>>>>>> A few suggestions:
>>>>>>>>>
>>>>>>>>> In your motherboard BIOS, do you have an option for primary 
>>>>>>>>> graphics to be toggled between on-board and PCIe?  If so, 
>>>>>>>>> please make sure that it’s set to NOT utilize the graphics 
>>>>>>>>> device you’re trying to use with the VM.  I’ve seen issues 
>>>>>>>>> with that consistently (most result in no GPU pass through at 
>>>>>>>>> all, but with some hardware, it may result in what you’re 
>>>>>>>>> experiencing).
>>>>>>>>>
>>>>>>>>> Also, are you using pci-stub to prevent the GPU from having a 
>>>>>>>>> nouveau driver loaded before starting your VM?  If not, this 
>>>>>>>>> may also impact your experience.
>>>>>>>>>
>>>>>>>>> I know we have several users passing through a GTX 750 without 
>>>>>>>>> difficulty, and without needing to load the rom file as you 
>>>>>>>>> are below.
>>>>>>>>>
>>>>>>>>> Hope this helps!
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>>
>>>>>>>>> Jonathan Panozzo
>>>>>>>>>
>>>>>>>>>> On Aug 28, 2015, at 10:42 AM, Marco Stagge 
>>>>>>>>>> <spiritblackdiamont at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello @all,
>>>>>>>>>>
>>>>>>>>>> at first, my english is not that good :/ so sorry for wrong 
>>>>>>>>>> spelling etc :o
>>>>>>>>>>
>>>>>>>>>> I've tried bunches of configs and hardware with kvm and 
>>>>>>>>>> passtrough, with Radeon Cards there are no problems but with 
>>>>>>>>>> Nvidia ...
>>>>>>>>>> it look like its nearly impossible ... it works to Display 
>>>>>>>>>> TianoCore bootscreen and Windows Install etc, but after 
>>>>>>>>>> installing the driver the whole machine freezes and on the 
>>>>>>>>>> console i got the message "kernel: disabling irq #20" wich is 
>>>>>>>>>> my GTX 750 Ti.
>>>>>>>>>>
>>>>>>>>>> the strange thingy is "x-vga=off" is the only way to see 
>>>>>>>>>> something on the monitor(i thought it should be "on" for 
>>>>>>>>>> this)....
>>>>>>>>>>
>>>>>>>>>> I've tried this with q35 chipset and the fx440, with seabios 
>>>>>>>>>> and TianoCore EFI, Windows 7 and Windows 10(dont have Win8 or 
>>>>>>>>>> want it ..)
>>>>>>>>>>
>>>>>>>>>> Here my config and specs
>>>>>>>>>>
>>>>>>>>>>    /root/qemu-2.4.0/x86_64-softmmu/qemu-system-x86_64 
>>>>>>>>>> -enable-kvm -M q35 -m 8G -cpu host,kvm=off \
>>>>>>>>>>    -smp 8,sockets=1,cores=8,threads=1 \
>>>>>>>>>>    -vga none -k de -rtc base=localtime -realtime mlock=on \
>>>>>>>>>>    -drive if=pflash,format=raw,file=/root/OVMF-pure-efi.fd \
>>>>>>>>>>    -monitor stdio \
>>>>>>>>>>    -debugcon file:/dev/stderr -global isa-debugcon.iobase=0x402 \
>>>>>>>>>>    -device virtio-scsi-pci,id=scsi \
>>>>>>>>>>    -device 
>>>>>>>>>> vfio-pci,host=02:00.0,rombar=1,x-vga=off,romfile=/root/GTX-750Ti.rom 
>>>>>>>>>> \
>>>>>>>>>>    -device vfio-pci,host=02:00.1 \
>>>>>>>>>>    -device vfio-pci,host=04:00.0 \
>>>>>>>>>>    -device vfio-pci,host=00:13.0 \
>>>>>>>>>>    -device vfio-pci,host=00:13.2 \
>>>>>>>>>>    -drive file=/dev/sdb,if=virtio,cache=none,discard=on \
>>>>>>>>>>    -drive 
>>>>>>>>>> file=/media/syno/Programme/windows/Windows_7_SP1_DE_AIO_X64.iso,id=isocd,if=none 
>>>>>>>>>> \
>>>>>>>>>>    -device scsi-cd,drive=isocd \
>>>>>>>>>>    -cdrom /media/syno/Programme/Linux/virtio-win-0.1-100.iso \
>>>>>>>>>>    -usbdevice host:04b4:8328 \
>>>>>>>>>>    -global isa-fdc.driveA=
>>>>>>>>>>
>>>>>>>>>> AMD FX8350
>>>>>>>>>> ASUS M5A99X EVO
>>>>>>>>>> 32GB Kingston Value RAM
>>>>>>>>>> AMD Radeon HD6670(guest) + Nvidia GTX 750 Ti(guest) + GT520(Host)
>>>>>>>>>> SSD now 120GB(guest) + Mushkin Chronos Deluxe 60GB(guest)
>>>>>>>>>>
>>>>>>>>>> interrupts:
>>>>>>>>>>            CPU0       CPU1       CPU2       CPU3       CPU4 
>>>>>>>>>> CPU5       CPU6       CPU7
>>>>>>>>>>   0:         47          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-IO-APIC-edge      timer
>>>>>>>>>>   1:          1          0          0          0 0          0 
>>>>>>>>>>          0          1  IR-IO-APIC-edge      i8042
>>>>>>>>>>   8:          0          0          0          0 0          0 
>>>>>>>>>>          1          0  IR-IO-APIC-edge      rtc0
>>>>>>>>>>   9:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-IO-APIC-fasteoi   acpi
>>>>>>>>>>  12:          0          1          0          1 0          2 
>>>>>>>>>>          0          0  IR-IO-APIC-edge      i8042
>>>>>>>>>>  17:      97731     573202      44343      68863     449791 
>>>>>>>>>> 278946     766521     145166  IR-IO-APIC   17-fasteoi 
>>>>>>>>>> ehci_hcd:usb1, vfio-intx(0000:0a:00.1)
>>>>>>>>>>  18:       6667      21540       5559       6748      24441 
>>>>>>>>>> 16748       4959       8611  IR-IO-APIC   18-fasteoi 
>>>>>>>>>> ohci_hcd:usb8, ohci_hcd:usb10
>>>>>>>>>>  19:      22052      93315      44652      67664     119162 
>>>>>>>>>> 75922      85567      23549  IR-IO-APIC   19-fasteoi 0000:00:11.0
>>>>>>>>>>  22:          0          3          1          8 2          1 
>>>>>>>>>>          1          1  IR-IO-APIC   22-fasteoi 
>>>>>>>>>> vfio-intx(0000:00:16.0)
>>>>>>>>>>  23:      10599      70971      15651      26344      78321 
>>>>>>>>>> 47939     232415      36216  IR-IO-APIC   23-fasteoi 
>>>>>>>>>> vfio-intx(0000:00:16.2)
>>>>>>>>>>  27:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge      AMD-Vi
>>>>>>>>>>  30:        859       2247    3874003     745023 1000 
>>>>>>>>>>        739       1462     145680  IR-PCI-MSI-edge      eth0
>>>>>>>>>>  33:     627986    4106418       3829       6174 1615 
>>>>>>>>>>        953        140        156  IR-PCI-MSI-edge 
>>>>>>>>>> vfio-msi[0](0000:0a:00.0)
>>>>>>>>>>  34:         43        740         54        294         43 
>>>>>>>>>> 4483073         43       1708  IR-PCI-MSI-edge 
>>>>>>>>>> vfio-msix[0](0000:07:00.0)
>>>>>>>>>>  35:          2          1          1         70 6         66 
>>>>>>>>>>          1      22716  IR-PCI-MSI-edge vfio-msix[1](0000:07:00.0)
>>>>>>>>>>  36:         41          1          0          6 3685144 
>>>>>>>>>>          2        118          2  IR-PCI-MSI-edge 
>>>>>>>>>> vfio-msix[2](0000:07:00.0)
>>>>>>>>>>  37:          1          1      20027          0 39 
>>>>>>>>>>          0        257          1  IR-PCI-MSI-edge 
>>>>>>>>>> vfio-msix[3](0000:07:00.0)
>>>>>>>>>>  38:       2617      16082        450        755     123855 
>>>>>>>>>> 75581      72374      11818  IR-PCI-MSI-edge 
>>>>>>>>>> vfio-msix[0](0000:06:00.0)
>>>>>>>>>>  39:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge vfio-msix[1](0000:06:00.0)
>>>>>>>>>>  40:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge vfio-msix[2](0000:06:00.0)
>>>>>>>>>>  41:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge vfio-msix[3](0000:06:00.0)
>>>>>>>>>>  42:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge vfio-msix[4](0000:06:00.0)
>>>>>>>>>>  43:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge vfio-msix[5](0000:06:00.0)
>>>>>>>>>>  44:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge vfio-msix[6](0000:06:00.0)
>>>>>>>>>>  45:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-PCI-MSI-edge vfio-msix[7](0000:06:00.0)
>>>>>>>>>>  51:       4115       5167       5148       5828       7122 
>>>>>>>>>> 5740      20944       7389  IR-IO-APIC   27-fasteoi 
>>>>>>>>>>   0000:05:00.0
>>>>>>>>>>  52:          0          0          0          0 0          0 
>>>>>>>>>>          0          0  IR-IO-APIC   23-fasteoi 0000:08:00.0
>>>>>>>>>>  54:      70872      71659      71958      71508      71437 
>>>>>>>>>> 72232      75968     115521  IR-PCI-MSI-edge      nouveau
>>>>>>>>>>  56:       5738       5576       5715       5605       5551 
>>>>>>>>>> 5403       5792      10014  IR-PCI-MSI-edge      nouveau
>>>>>>>>>> NMI:        535        629        505        409 474 
>>>>>>>>>>        508        457        388   Non-maskable interrupts
>>>>>>>>>> LOC:    7873279    2105376    6840648    3347764    5748321 
>>>>>>>>>> 2139094    7856471    3305521   Local timer interrupts
>>>>>>>>>> SPU:          0          0          0          0 0          0 
>>>>>>>>>>          0          0   Spurious interrupts
>>>>>>>>>> PMI:        535        629        505        409 474 
>>>>>>>>>>        508        457        388   Performance monitoring 
>>>>>>>>>> interrupts
>>>>>>>>>> IWI:          0          0          0          0 0          1 
>>>>>>>>>>          0          0   IRQ work interrupts
>>>>>>>>>> RTR:          0          0          0          0 0          0 
>>>>>>>>>>          0          0   APIC ICR read retries
>>>>>>>>>> RES:   31834176   46456464   40752536   13038581   29382846 
>>>>>>>>>> 31897150   34234584   16740944   Rescheduling interrupts
>>>>>>>>>> CAL:       2026       1990       2018       2081       2046 
>>>>>>>>>> 1441       2141       2063   Function call interrupts
>>>>>>>>>> TLB:       3857       3712       3424       6321       4082 
>>>>>>>>>> 4559       3763       6583   TLB shootdowns
>>>>>>>>>> TRM:          0          0          0          0 0          0 
>>>>>>>>>>          0          0   Thermal event interrupts
>>>>>>>>>> THR:          0          0          0          0 0          0 
>>>>>>>>>>          0          0   Threshold APIC interrupts
>>>>>>>>>> MCE:          0          0          0          0 0          0 
>>>>>>>>>>          0          0   Machine check exceptions
>>>>>>>>>> MCP:         83         83         83         83 83 
>>>>>>>>>>         83         83         83   Machine check polls
>>>>>>>>>> HYP:          0          0          0          0 0          0 
>>>>>>>>>>          0          0   Hypervisor callback interrupts
>>>>>>>>>> ERR:          0
>>>>>>>>>> MIS:          0
>>>>>>>>>>
>>>>>>>>>> any ideas?
>>>>>>>>>>
>>>>>>>>>> greetings
>>>>>>>>>> Spirit
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> vfio-users mailing list
>>>>>>>>>> vfio-users at redhat.com <mailto:vfio-users at redhat.com>
>>>>>>>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150828/afe65b81/attachment.htm>


More information about the vfio-users mailing list