[vfio-users] can't get my video card to connect to my vm

[Stein van Broekhoven]

Result did not change after following the guide:

06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Turks XT [Radeon HD 6670/7670] [1002:6758] (prog-if 00 [VGA controller])
        Subsystem: ASUSTeK Computer Inc. Device [1043:03ea]
        Flags: fast devsel, IRQ 55, NUMA node 0
        Memory at b0000000 (64-bit, prefetchable) [size=256M]
        Memory at fe520000 (64-bit, non-prefetchable) [size=128K]
        I/O ports at b000 [size=256]
        Expansion ROM at fe500000 [disabled] [size=128K]
        Capabilities: <access denied>
        Kernel driver in use: vfio-pci
        Kernel modules: radeon, fglrx

06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Turks/Whistler HDMI Audio [Radeon HD 6000 Series] [1002:aa90]
        Subsystem: ASUSTeK Computer Inc. Device [1043:aa90]
        Flags: bus master, fast devsel, latency 0, IRQ 7, NUMA node 0
        Memory at fe540000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

dmesg:
[   40.189677] genirq: Flags mismatch irq 55. 00000080 (vfio-intx(0000:06:00.0)) vs. 00000800 (fglrx[1]@PCI:6:0:0)

qemu start:
qemu-system-x86_64: -device vfio-pci,host=06:00.0,addr=09.0,multifunction=on: vfio: Error: Failed to setup INTx fd: Device or resource busy
qemu-system-x86_64: -device vfio-pci,host=06:00.0,addr=09.0,multifunction=on: Device initialization failed

/proc/interrupts:
  54:      24036      23881      23914      23982      24003      24049      24762      32248   PCI-MSI-edge      fglrx[0]@PCI:1:0:0
  55:          0          0          0          0          0          0          0          0   PCI-MSI-edge      fglrx[1]@PCI:6:0:0



----- Oorspronkelijk bericht -----
Van: "Alex Williamson" <alex.l.williamson at gmail.com>
Aan: "Stein van Broekhoven" <stein at aapjeisbaas.nl>
Cc: "vfio-users" <vfio-users at redhat.com>
Verzonden: Dinsdag 8 maart 2016 23:26:18
Onderwerp: Re: [vfio-users] can't get my video card to connect to my vm

On Tue, Mar 8, 2016 at 3:20 PM, Stein van Broekhoven < stein at aapjeisbaas.nl > wrote: 


I tried 2 things to avoid: "fglrx driver claiming the interrupt" 
Start vm before fglrx gets to claiming it. (bad idea!) 

Edit xorg related config to see what is connecting the driver to the card. (better idea) 
All i found was PCI:1:0:0 (first GPU) in the config 
Also tried "# aticonfig --adapter=0 --initial" and got the same config as my system created for me. 

Any ideas on how i can keep the following process from claiming it? 
dmesg: 
[ 12.361476] <6>[fglrx] Firegl kernel thread PID: 659 
[ 12.361547] <6>[fglrx] Firegl kernel thread PID: 660 
[ 12.361616] <6>[fglrx] Firegl kernel thread PID: 661 
[ 12.361730] <6>[fglrx] IRQ 55 Enabled 
[ 12.439300] <6>[fglrx] Reserved FB block: Shared offset:0, size:1000000 
[ 12.439302] <6>[fglrx] Reserved FB block: Unshared offset:fab4000, size:4000 
[ 12.439303] <6>[fglrx] Reserved FB block: Unshared offset:fab8000, size:548000 

ps faux: 
root 2 0.0 0.0 0 0 ? S 22:53 0:00 [kthreadd] 
... 
root 659 0.0 0.0 0 0 ? S 22:53 0:00 \_ [firegl] 
... 


As I said before, you need to use pci-stub or vfio-pci to prevent fglrx from claiming the device, follow something like this guide - http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-3-host.html 

I can't guess why trying to start the vm before fglrx was a "bad idea" in your first case, getting pci-stub or vfio-pci bound to the device before fglrx can break it is usually the recommended practice. In the simplest case, just add pci-stub.ids=1002:6758 to your kernel commandline.