[vfio-users] KVM Nvidia Passtrough Code 43 and freezes
Jon Panozzo
jonp at lime-technology.com
Fri Aug 28 19:48:54 UTC 2015
re: crash, try the new virtio drivers. could be the crash is unrelated to vfio/graphics assignment. As far as command line for cpuset, I honestly don’t know because I always use libvirt/xml to accomplish that.
> On Aug 28, 2015, at 1:39 PM, Marco Stagge <spiritblackdiamont at gmail.com> wrote:
>
> Hmm i need some tuning, the driver just crashed at steam install :o
>
> 1 - i know how to do that on xml, but howto on command line?
>
> 2 - ah thank didnt notice there are new virtio drivers, thanks :)
>
> greetings
> Marco
>
> Am 28.08.2015 um 20:15 schrieb Jon Panozzo:
>> Awesome! I think you’ll be happy with your benchmark scores, but to improve them further, I’d suggest a few additional tweaks:
>>
>> 1 - cpuset. you should pin the virtual CPUs in your VM to physical threads available on the host (e.g. make vcpu 0 map to pcpu 0). This reduces unnecessary overhead caused by context switching and can have a pretty big impact to performance in physics benchmarks (this applies to gaming).
>>
>> 2 - I’d also consider using a more current version of the virtio drivers. There were a few bugs with Windows 10 discovered that have since been patched. The “latest” drivers available from the Fedora project are 0.1.109.
>>
>> Glad you got things working!
>>
>> - Jon
>>
>>> On Aug 28, 2015, at 1:11 PM, Marco Stagge < <mailto:spiritblackdiamont at gmail.com>spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>
>>> Uhm yeah, i dont know how i did this but i think i was the initramfs......
>>>
>>> i checked lspci -k on my installation, and the 750ti wasnt assigned to pci-stub.....after checking the initramfs-tools/modules, i tried to update the initramfs(i thought i did forget this) and debian throw me a strange error.... after googlin around and solving this problem, i rebootet my system and reinstalled my win10 vm.
>>>
>>> The devicemanager display no error on the 750ti and i'm on fullhd atm :)
>>> The nvidia panel runs without problems, driver is 353.54 :)
>>>
>>> Well thank you all, guys :)
>>> I'll do a benchmark etc now, yay ^_^
>>>
>>> greetings
>>> Marco
>>>
>>> Am 28.08.2015 um 19:31 schrieb Jon Panozzo:
>>>> Hmm. Can you ssh into the system and type lspci -k and copy/paste the results back here? This is very odd behavior indeed.
>>>>
>>>> One other thing we can try too is to pci-stub this device and reboot. Not sure how you were doing it on your system, but with unRAID, you do this by logging in via ssh, then running the command lspci -n. Locate the row for your device (the 02:00.0 and .1), then copy the vendor/product ID.
>>>>
>>>> Back on the webGui, go to the Main tab, then click on the word “Flash” under the devices column. Now go to the syslinux section and modify the /append line. Here’s mine, where I stub numerous devices:
>>>>
>>>> default /syslinux/menu.c32
>>>> menu title Lime Technology
>>>> prompt 0
>>>> timeout 50
>>>> label unRAID OS
>>>> menu default
>>>> kernel /bzimage
>>>> append pci-stub.ids=1002:68f9,1002:aa68,10de:1004,10de:0e1a,1a0a:6202,8086:0412 pcie_acs_override=downstream initrd=/bzroot
>>>> label unRAID OS Safe Mode (no plugins)
>>>> kernel /bzimage
>>>> append initrd=/bzroot unraidsafemode
>>>> label Memtest86+
>>>> kernel /memtest
>>>> label Xen/unRAID OS
>>>> kernel /syslinux/mboot.c32
>>>> append /xen dom0_mem=4096M iommu=on --- /bzimage xen-pciback.hide=(01:00.0)(01:00.1)(02:00.0)(02:00.1) intel_iommu=on --- /bzroot
>>>> label Xen/unRAID OS Safe Mode (no plugins)
>>>> kernel /syslinux/mboot.c32
>>>> append /xen --- /bzimage --- /bzroot unraidsafemode
>>>>
>>>> You shouldn’t need to stub the devices in unRAID because we don’t load GPU drivers, so nothing should be bound to the device.
>>>>
>>>> If neither of these work, you may have me stumped :-(
>>>>
>>>> As a last resort, you can go to Tools -> Diagnostics and click collect, then send me that zip file for further analysis.
>>>>
>>>> - Jon
>>>>
>>>>> On Aug 28, 2015, at 12:13 PM, Marco Stagge <spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>>
>>>>> okay here:
>>>>>
>>>>> /sys/kernel/iommu_groups/0/devices/0000:00:00.0
>>>>> /sys/kernel/iommu_groups/1/devices/0000:00:02.0
>>>>> /sys/kernel/iommu_groups/2/devices/0000:00:03.0
>>>>> /sys/kernel/iommu_groups/3/devices/0000:00:04.0
>>>>> /sys/kernel/iommu_groups/4/devices/0000:00:05.0
>>>>> /sys/kernel/iommu_groups/5/devices/0000:00:06.0
>>>>> /sys/kernel/iommu_groups/6/devices/0000:00:07.0
>>>>> /sys/kernel/iommu_groups/7/devices/0000:00:09.0
>>>>> /sys/kernel/iommu_groups/8/devices/0000:00:0a.0
>>>>> /sys/kernel/iommu_groups/9/devices/0000:00:11.0
>>>>> /sys/kernel/iommu_groups/10/devices/0000:00:12.0
>>>>> /sys/kernel/iommu_groups/10/devices/0000:00:12.2
>>>>> /sys/kernel/iommu_groups/11/devices/0000:00:13.0
>>>>> /sys/kernel/iommu_groups/11/devices/0000:00:13.2
>>>>> /sys/kernel/iommu_groups/12/devices/0000:00:14.0
>>>>> /sys/kernel/iommu_groups/13/devices/0000:00:14.2
>>>>> /sys/kernel/iommu_groups/14/devices/0000:00:14.3
>>>>> /sys/kernel/iommu_groups/15/devices/0000:00:14.4
>>>>> /sys/kernel/iommu_groups/15/devices/0000:09:06.0
>>>>> /sys/kernel/iommu_groups/16/devices/0000:00:14.5
>>>>> /sys/kernel/iommu_groups/17/devices/0000:00:15.0
>>>>> /sys/kernel/iommu_groups/18/devices/0000:00:16.0
>>>>> /sys/kernel/iommu_groups/18/devices/0000:00:16.2
>>>>> /sys/kernel/iommu_groups/19/devices/0000:01:00.0
>>>>> /sys/kernel/iommu_groups/19/devices/0000:01:00.1
>>>>> /sys/kernel/iommu_groups/20/devices/0000:02:00.0
>>>>> /sys/kernel/iommu_groups/20/devices/0000:02:00.1
>>>>> /sys/kernel/iommu_groups/21/devices/0000:03:00.0
>>>>> /sys/kernel/iommu_groups/22/devices/0000:04:00.0
>>>>> /sys/kernel/iommu_groups/23/devices/0000:05:00.0
>>>>> /sys/kernel/iommu_groups/24/devices/0000:06:00.0
>>>>> /sys/kernel/iommu_groups/25/devices/0000:07:00.0
>>>>> /sys/kernel/iommu_groups/26/devices/0000:08:00.0
>>>>> /sys/kernel/iommu_groups/27/devices/0000:0a:00.0
>>>>> /sys/kernel/iommu_groups/27/devices/0000:0a:00.1
>>>>>
>>>>>
>>>>> Am 28.08.2015 um 19:11 schrieb Jon Panozzo:
>>>>>> Under Tools -> System Devices, please copy and paste the IOMMU groups section back in an e-mail to me.
>>>>>>
>>>>>> - Jon
>>>>>>
>>>>>>> On Aug 28, 2015, at 12:09 PM, Marco Stagge < <mailto:spiritblackdiamont at gmail.com>spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>>>>
>>>>>>> done, but still the same error :(
>>>>>>>
>>>>>>> Am 28.08.2015 um 18:59 schrieb Jon Panozzo:
>>>>>>>> Ok, that is because you need to enable PCI ACS Override. Under Settings -> VM Manager, enable PCI ACS Override. Then reboot your system.
>>>>>>>>
>>>>>>>>> On Aug 28, 2015, at 11:57 AM, Marco Stagge < <mailto:spiritblackdiamont at gmail.com>spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Jon,
>>>>>>>>>
>>>>>>>>> i've tried to create a vm but its not possible:
>>>>>>>>> Error creating VM: internal error: early end of file from monitor: possible problem:
>>>>>>>>> 2015-08-28T16:54:50.210851Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted
>>>>>>>>> 2015-08-28T16:54:50.210877Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 20
>>>>>>>>> 2015-08-28T16:54:50.210886Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 20
>>>>>>>>> 2015-08-28T16:54:50.210898Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed
>>>>>>>>> 2015-08-28T16:54:50.210910Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized
>>>>>>>>>
>>>>>>>>> did i missed something?
>>>>>>>>>
>>>>>>>>> greetings
>>>>>>>>> Marco
>>>>>>>>>
>>>>>>>>> Am 28.08.2015 um 18:05 schrieb Jon Panozzo:
>>>>>>>>>> Hi Marco,
>>>>>>>>>>
>>>>>>>>>> Not sure if you want to go down this road or not, but a relatively quick path to checking to see if this is a kernel/OS config issue would be to load up our OS (unRAID) which boots of a USB stick and see if you have success there. We’ve tuned our OS for use with VFIO/VGA assignment, and we don’t load a desktop GUI at all (nor any graphics devices). The OS uses libvirt and provides a webGui for testing various configs (OVMF vs. SeaBIOS and i440fx vs. Q35). Hopefully this isn’t seen as a shameless plug on our OS here, but it could be a way to narrow down the issue, and it’s free to try ( <http://lime-technology.com/>http://lime-technology.com <http://lime-technology.com/>).
>>>>>>>>>>
>>>>>>>>>> I can tell you that the NVIDIA drivers have nothing to do with it. I have multiple NVIDIA GPUs that are running on multiple systems. So long as you don’t enable the hyper-v enlightenments through QEMU/KVM and specify kvm=off like you are, the card does not detect that it is running in a virtual machine / render the code 43 error. There are other issues that can cause code 43 unrelated to detection by the device drivers, but narrowing down the specifics can be a challenge.
>>>>>>>>>>
>>>>>>>>>> - Jon
>>>>>>>>>>
>>>>>>>>>>> On Aug 28, 2015, at 10:58 AM, Marco Stagge < <mailto:spiritblackdiamont at gmail.com>spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Jonathan,
>>>>>>>>>>>
>>>>>>>>>>> this is a AMD Board without a integrated GPU :)
>>>>>>>>>>>
>>>>>>>>>>> Yep, nouveau driver don't load on this card but the nouveau driver tells me on boot some strange lines at dmesg but can display the EFI Screen etc.
>>>>>>>>>>>
>>>>>>>>>>> Yeah i saw on the bbs from archlinux some guys with nvidia cards running, but this thread is a bit "old". So i think nvidia changed the driver to recognize the machine as virtual(Windows 10 does).
>>>>>>>>>>>
>>>>>>>>>>> Without the romfile, it wont display anything(same on the radeon cards) :o
>>>>>>>>>>>
>>>>>>>>>>> Here's my lspci -k:
>>>>>>>>>>> 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B)
>>>>>>>>>>> 00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
>>>>>>>>>>> 00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port B)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port C)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port D)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port E)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port F)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port G)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port H)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx1 port A)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. Device 84dd
>>>>>>>>>>> Kernel driver in use: ahci
>>>>>>>>>>> 00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>>> Kernel driver in use: ohci-pci
>>>>>>>>>>> 00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>>> Kernel driver in use: ehci-pci
>>>>>>>>>>> 00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
>>>>>>>>>>> Kernel driver in use: piix4_smbus
>>>>>>>>>>> 00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) (rev 40)
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. M4A89GTD PRO/USB3 Motherboard
>>>>>>>>>>> 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
>>>>>>>>>>> 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
>>>>>>>>>>> 00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
>>>>>>>>>>> Kernel driver in use: ohci-pci
>>>>>>>>>>> 00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
>>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>>> 00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
>>>>>>>>>>> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
>>>>>>>>>>> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
>>>>>>>>>>> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
>>>>>>>>>>> Kernel driver in use: k10temp
>>>>>>>>>>> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
>>>>>>>>>>> Kernel driver in use: fam15h_power
>>>>>>>>>>> 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
>>>>>>>>>>> 01:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 520] (rev a1)
>>>>>>>>>>> Subsystem: ZOTAC International (MCO) Ltd. Device 3214
>>>>>>>>>>> Kernel driver in use: nouveau
>>>>>>>>>>> 01:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)
>>>>>>>>>>> Subsystem: ZOTAC International (MCO) Ltd. Device 3214
>>>>>>>>>>> 02:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] (rev a2)
>>>>>>>>>>> Subsystem: eVga.com <http://evga.com/>. Corp. Device 3753
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 02:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)
>>>>>>>>>>> Subsystem: eVga.com <http://evga.com/>. Corp. Device 3753
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8P67 and other motherboards
>>>>>>>>>>> Kernel driver in use: r8169
>>>>>>>>>>> 04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 05:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA Controller (rev 10)
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard
>>>>>>>>>>> Kernel driver in use: ahci
>>>>>>>>>>> 06:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
>>>>>>>>>>> Subsystem: Device 7470:3468
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 08:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA Controller (rev 10)
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard
>>>>>>>>>>> Kernel driver in use: ahci
>>>>>>>>>>> 09:06.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)
>>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. Motherboard
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 0a:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Turks XT [Radeon HD 6670/7670]
>>>>>>>>>>> Subsystem: PC Partner Limited / Sapphire Technology Device e195
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>> 0a:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Turks/Whistler HDMI Audio [Radeon HD 6000 Series]
>>>>>>>>>>> Subsystem: PC Partner Limited / Sapphire Technology Device aa90
>>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>>
>>>>>>>>>>> greetings
>>>>>>>>>>> Marco - Spirit
>>>>>>>>>>>
>>>>>>>>>>> Am 28.08.2015 um 17:49 schrieb Jon Panozzo:
>>>>>>>>>>>> Hi Marco,
>>>>>>>>>>>>
>>>>>>>>>>>> A few suggestions:
>>>>>>>>>>>>
>>>>>>>>>>>> In your motherboard BIOS, do you have an option for primary graphics to be toggled between on-board and PCIe? If so, please make sure that it’s set to NOT utilize the graphics device you’re trying to use with the VM. I’ve seen issues with that consistently (most result in no GPU pass through at all, but with some hardware, it may result in what you’re experiencing).
>>>>>>>>>>>>
>>>>>>>>>>>> Also, are you using pci-stub to prevent the GPU from having a nouveau driver loaded before starting your VM? If not, this may also impact your experience.
>>>>>>>>>>>>
>>>>>>>>>>>> I know we have several users passing through a GTX 750 without difficulty, and without needing to load the rom file as you are below.
>>>>>>>>>>>>
>>>>>>>>>>>> Hope this helps!
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>
>>>>>>>>>>>> Jonathan Panozzo
>>>>>>>>>>>>
>>>>>>>>>>>>> On Aug 28, 2015, at 10:42 AM, Marco Stagge < <mailto:spiritblackdiamont at gmail.com>spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello @all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> at first, my english is not that good :/ so sorry for wrong spelling etc :o
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've tried bunches of configs and hardware with kvm and passtrough, with Radeon Cards there are no problems but with Nvidia ...
>>>>>>>>>>>>> it look like its nearly impossible ... it works to Display TianoCore bootscreen and Windows Install etc, but after installing the driver the whole machine freezes and on the console i got the message "kernel: disabling irq #20" wich is my GTX 750 Ti.
>>>>>>>>>>>>>
>>>>>>>>>>>>> the strange thingy is "x-vga=off" is the only way to see something on the monitor(i thought it should be "on" for this)....
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've tried this with q35 chipset and the fx440, with seabios and TianoCore EFI, Windows 7 and Windows 10(dont have Win8 or want it ..)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here my config and specs
>>>>>>>>>>>>>
>>>>>>>>>>>>> /root/qemu-2.4.0/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -M q35 -m 8G -cpu host,kvm=off \
>>>>>>>>>>>>> -smp 8,sockets=1,cores=8,threads=1 \
>>>>>>>>>>>>> -vga none -k de -rtc base=localtime -realtime mlock=on \
>>>>>>>>>>>>> -drive if=pflash,format=raw,file=/root/OVMF-pure-efi.fd \
>>>>>>>>>>>>> -monitor stdio \
>>>>>>>>>>>>> -debugcon <file:///dev/stderr>file:/dev/stderr <file:///dev/stderr> -global isa-debugcon.iobase=0x402 \
>>>>>>>>>>>>> -device virtio-scsi-pci,id=scsi \
>>>>>>>>>>>>> -device vfio-pci,host=02:00.0,rombar=1,x-vga=off,romfile=/root/GTX-750Ti.rom \
>>>>>>>>>>>>> -device vfio-pci,host=02:00.1 \
>>>>>>>>>>>>> -device vfio-pci,host=04:00.0 \
>>>>>>>>>>>>> -device vfio-pci,host=00:13.0 \
>>>>>>>>>>>>> -device vfio-pci,host=00:13.2 \
>>>>>>>>>>>>> -drive file=/dev/sdb,if=virtio,cache=none,discard=on \
>>>>>>>>>>>>> -drive file=/media/syno/Programme/windows/Windows_7_SP1_DE_AIO_X64.iso,id=isocd,if=none \
>>>>>>>>>>>>> -device scsi-cd,drive=isocd \
>>>>>>>>>>>>> -cdrom /media/syno/Programme/Linux/virtio-win-0.1-100.iso \
>>>>>>>>>>>>> -usbdevice host:04b4:8328 \
>>>>>>>>>>>>> -global isa-fdc.driveA=
>>>>>>>>>>>>>
>>>>>>>>>>>>> AMD FX8350
>>>>>>>>>>>>> ASUS M5A99X EVO
>>>>>>>>>>>>> 32GB Kingston Value RAM
>>>>>>>>>>>>> AMD Radeon HD6670(guest) + Nvidia GTX 750 Ti(guest) + GT520(Host)
>>>>>>>>>>>>> SSD now 120GB(guest) + Mushkin Chronos Deluxe 60GB(guest)
>>>>>>>>>>>>>
>>>>>>>>>>>>> interrupts:
>>>>>>>>>>>>> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
>>>>>>>>>>>>> 0: 47 0 0 0 0 0 0 0 IR-IO-APIC-edge timer
>>>>>>>>>>>>> 1: 1 0 0 0 0 0 0 1 IR-IO-APIC-edge i8042
>>>>>>>>>>>>> 8: 0 0 0 0 0 0 1 0 IR-IO-APIC-edge rtc0
>>>>>>>>>>>>> 9: 0 0 0 0 0 0 0 0 IR-IO-APIC-fasteoi acpi
>>>>>>>>>>>>> 12: 0 1 0 1 0 2 0 0 IR-IO-APIC-edge i8042
>>>>>>>>>>>>> 17: 97731 573202 44343 68863 449791 278946 766521 145166 IR-IO-APIC 17-fasteoi ehci_hcd:usb1, vfio-intx(0000:0a:00.1)
>>>>>>>>>>>>> 18: 6667 21540 5559 6748 24441 16748 4959 8611 IR-IO-APIC 18-fasteoi ohci_hcd:usb8, ohci_hcd:usb10
>>>>>>>>>>>>> 19: 22052 93315 44652 67664 119162 75922 85567 23549 IR-IO-APIC 19-fasteoi 0000:00:11.0
>>>>>>>>>>>>> 22: 0 3 1 8 2 1 1 1 IR-IO-APIC 22-fasteoi vfio-intx(0000:00:16.0)
>>>>>>>>>>>>> 23: 10599 70971 15651 26344 78321 47939 232415 36216 IR-IO-APIC 23-fasteoi vfio-intx(0000:00:16.2)
>>>>>>>>>>>>> 27: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge AMD-Vi
>>>>>>>>>>>>> 30: 859 2247 3874003 745023 1000 739 1462 145680 IR-PCI-MSI-edge eth0
>>>>>>>>>>>>> 33: 627986 4106418 3829 6174 1615 953 140 156 IR-PCI-MSI-edge vfio-msi[0](0000:0a:00.0)
>>>>>>>>>>>>> 34: 43 740 54 294 43 4483073 43 1708 IR-PCI-MSI-edge vfio-msix[0](0000:07:00.0)
>>>>>>>>>>>>> 35: 2 1 1 70 6 66 1 22716 IR-PCI-MSI-edge vfio-msix[1](0000:07:00.0)
>>>>>>>>>>>>> 36: 41 1 0 6 3685144 2 118 2 IR-PCI-MSI-edge vfio-msix[2](0000:07:00.0)
>>>>>>>>>>>>> 37: 1 1 20027 0 39 0 257 1 IR-PCI-MSI-edge vfio-msix[3](0000:07:00.0)
>>>>>>>>>>>>> 38: 2617 16082 450 755 123855 75581 72374 11818 IR-PCI-MSI-edge vfio-msix[0](0000:06:00.0)
>>>>>>>>>>>>> 39: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge vfio-msix[1](0000:06:00.0)
>>>>>>>>>>>>> 40: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge vfio-msix[2](0000:06:00.0)
>>>>>>>>>>>>> 41: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge vfio-msix[3](0000:06:00.0)
>>>>>>>>>>>>> 42: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge vfio-msix[4](0000:06:00.0)
>>>>>>>>>>>>> 43: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge vfio-msix[5](0000:06:00.0)
>>>>>>>>>>>>> 44: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge vfio-msix[6](0000:06:00.0)
>>>>>>>>>>>>> 45: 0 0 0 0 0 0 0 0 IR-PCI-MSI-edge vfio-msix[7](0000:06:00.0)
>>>>>>>>>>>>> 51: 4115 5167 5148 5828 7122 5740 20944 7389 IR-IO-APIC 27-fasteoi 0000:05:00.0
>>>>>>>>>>>>> 52: 0 0 0 0 0 0 0 0 IR-IO-APIC 23-fasteoi 0000:08:00.0
>>>>>>>>>>>>> 54: 70872 71659 71958 71508 71437 72232 75968 115521 IR-PCI-MSI-edge nouveau
>>>>>>>>>>>>> 56: 5738 5576 5715 5605 5551 5403 5792 10014 IR-PCI-MSI-edge nouveau
>>>>>>>>>>>>> NMI: 535 629 505 409 474 508 457 388 Non-maskable interrupts
>>>>>>>>>>>>> LOC: 7873279 2105376 6840648 3347764 5748321 2139094 7856471 3305521 Local timer interrupts
>>>>>>>>>>>>> SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
>>>>>>>>>>>>> PMI: 535 629 505 409 474 508 457 388 Performance monitoring interrupts
>>>>>>>>>>>>> IWI: 0 0 0 0 0 1 0 0 IRQ work interrupts
>>>>>>>>>>>>> RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries
>>>>>>>>>>>>> RES: 31834176 46456464 40752536 13038581 29382846 31897150 34234584 16740944 Rescheduling interrupts
>>>>>>>>>>>>> CAL: 2026 1990 2018 2081 2046 1441 2141 2063 Function call interrupts
>>>>>>>>>>>>> TLB: 3857 3712 3424 6321 4082 4559 3763 6583 TLB shootdowns
>>>>>>>>>>>>> TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
>>>>>>>>>>>>> THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
>>>>>>>>>>>>> MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
>>>>>>>>>>>>> MCP: 83 83 83 83 83 83 83 83 Machine check polls
>>>>>>>>>>>>> HYP: 0 0 0 0 0 0 0 0 Hypervisor callback interrupts
>>>>>>>>>>>>> ERR: 0
>>>>>>>>>>>>> MIS: 0
>>>>>>>>>>>>>
>>>>>>>>>>>>> any ideas?
>>>>>>>>>>>>>
>>>>>>>>>>>>> greetings
>>>>>>>>>>>>> Spirit
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> vfio-users mailing list
>>>>>>>>>>>>> <mailto:vfio-users at redhat.com>vfio-users at redhat.com <mailto:vfio-users at redhat.com>
>>>>>>>>>>>>> <https://www.redhat.com/mailman/listinfo/vfio-users>https://www.redhat.com/mailman/listinfo/vfio-users <https://www.redhat.com/mailman/listinfo/vfio-users>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150828/52443c10/attachment.htm>
More information about the vfio-users
mailing list