[vfio-users] Need help with GPU Passthrough on Ryzen C6H + GTX 980 Ti + GTX 1060 6G

[Alex Williamson]

On Wed, Jul 5, 2017 at 9:10 PM, Thiago Ramon <thiagoramon at gmail.com> wrote:

> I'm having a quite unique problem, and have exhausted all possibilities I
> have found so far, after a couple weeks of attempts, so I've decided to
> bother you guys with this.
>
> My setup: Ryzen 7 1800X, Asus Crosshair VI Hero, NVidia GTX 980 Ti and
> NVidia GTX 1060 6G
> OS: Arch Linux, latest updates, mainline kernel (4.11.7-1-ARCH),
> QEMU 2.9.0, libvirt 3.4.0
>
> The problem, for all I can tell, is that the GPU is getting corrupted
> somehow at or before reaching the BIOS/UEFI, getting reset and stuck on
> mode D3, no matter which GPU I passthrough, boot options, SeaBIOS/OVMF,
> chipset or connection to the PCI/PCIe bus.
>
> Both GPUs are healthy and working perfectly under Linux, using the
> proprietary NVidia drivers.
>
> Things tried: Disabled D3 mode in vfio_pci, used pci_stub instead,
> disabled NVidia driver and ran the VM from the console, multiple boot
> options involving the IOMMU and KVM (but hey, any new ideas help)
>
> I know the motherboard (in general, not this one specifically) can work
> with GPU passthrough, as I already had contact with someone passing a GTX
> 1070 with it (though his other GPU is AMD).
>
> Unless there's something I've overlooked, I probably need to gather more
> in-depth information on what's going on with the GPU in the first moments
> of the boot process, so if anyone knows of a good set of debug options for
> QEMU, or if kernel tracing is better, please let me know.
>
> Thanks for any help, and let me know if there's any extra info that could
> help solve this puzzle.
>
>
> Relevant logs and more details: https://www.reddit.
> com/r/VFIO/comments/6khu5i/need_help_with_gpu_
> passthrough_on_ryzen_c6h_gtx/
>

Wow, formatting in reddit is nearly impossible to decipher... pastebin?  I
can spot one issue:

 pci 0000:29:00.0: vgaarb: setting as boot VGA device

Generally you want to assign the non-boot device.  And probably related:

Failed to mmap 0000:29:00.0 BAR 3. Performance may be slow

This is really suggesting something much more wrong than performance may be
slow.  Check /proc/iomem, find what driver is claiming resources on the
device, disable it.  This probably means that some other driver besides
vesafb or efifb is blocking the device.  The kernel will try pretty hard to
attach a driver to the primary graphics, which is one of the complications
of trying to assign primary graphics.  Thanks,

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20170705/0a55c63c/attachment.htm>