[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[vfio-users] No video output from dedicated GPU when doing PCI passthrough



Hi!

I've been trying to get PCI/GPU passthrough to work for a few days but I've hit a wall. I can't get any video output from the passed GPU. I'm trying to virtualize Windows 10 Home Edition. I've been using the following guides:

 - Arch Wiki: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF
- Reddit: https://www.reddit.com/r/linuxmasterrace/comments/2z30dq/gpu_passthrough_or_how_to_play_any_game_at_near/?st=j1kioetn&sh=a198702b
 - http://vfio.blogspot.com/
- Level1Techs: https://forum.level1techs.com/t/play-games-in-windows-on-linux-pci-passthrough-quick-guide/108981

My hardware should support this:

 - OS: Arch Linux with kernel version 4.10.9-1-ARCH
 - CPU: Intel Core i5-6600K (supports VT-d and VT-x)
 - GPU: Intel HD Graphics 530 (iGPU, used by Linux)
 - GPU: AMD Radeon R9 380 (passed through)
- Mobo: Asus Z170 Pro Gaming (if I'm interpreting the manual correctly, hardware virtualization should be supported)
 - RAM: 2x4GB Kingston Savage DDR4-3000

VT-d is enabled from the UEFI BIOS and IOMMU is enabled in the kernel. This can be confirmed with `dmesg | grep -i iommu`:

[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=c6c3c8f3-030c-4dce-be63-8938386ac963 rw quiet intel_iommu=on [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=c6c3c8f3-030c-4dce-be63-8938386ac963 rw quiet intel_iommu=on
    [    0.000000] DMAR: IOMMU enabled
    ...

The Radeon GPU and its HDMI audio controller are using the vfio-pci driver and are in the IOMMU group 1 with a PCIe controller. Output of `lspci -nnk`:

    ...
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Tonga PRO [Radeon R9 285/380] [1002:6939] (rev f1) Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:2015]
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Tonga HDMI Audio [Radeon R9 285/380] [1002:aad8] Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:aad8]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
    ...

vfio-pci is running properly, as proved by `dmesg | grep -i vfio`:

    [    0.773952] VFIO - User Level meta-driver version: 0.3
[ 0.774978] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none [ 0.789351] vfio_pci: add [1002:6939[ffff:ffff]] class 0x000000/00000000 [ 0.806016] vfio_pci: add [1002:aad8[ffff:ffff]] class 0x000000/00000000 [ 2.767507] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
    [ 1310.412510] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)

QEMU, libvirt, OVMF and virt-manager are installed and libvirtd-daemon is running. Output of `systemctl status libvirtd`:

    ● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: disabled)
       Active: active (running) since Sat 2017-04-15 12:16:01 EEST; 24h ago
         Docs: man:libvirtd(8)
               http://libvirt.org
     Main PID: 469 (libvirtd)
        Tasks: 21 (limit: 4915)
       CGroup: /system.slice/libvirtd.service
               ├─469 /usr/bin/libvirtd
├─708 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper ├─709 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper ├─754 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/wireless.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper └─755 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/wireless.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper

I setup my VM with virt-manager like this (exact steps):

 1. Create a new QEMU/KVM VM from local install media
2. Use a Windows 10 ISO image found from the hard disk. OS type and version are detected properly.
 3. Use default settings for RAM and CPU: 4096 MiB and 2 cores
4. Create custom storage in /opt/os-storage/win10.qcow2 with size of 60 GiB
 5. Check "Customize configuration before install" and click Finish
 6. In "Overview":
    - Change Firmware to UEFI x86_64: /usr/share/ovmf/x64/ovmf_x64.bin
    - Chipset is "i440FX" and Architecture "x86_64"
 7. In "CPUs":
    - Change Model to "host-passthrough"
 8. In "IDE Disk 1":
    - Change Disk bus to "VirtIO"
    - Change Cache mode to "writeback"
 9. In "IDE CDROM 1":
    - Connect it to the Win10 ISO
    - Change Disk bus to "SATA"
 10. In "NIC ...":
    - Change Device model to "virtio"
 11. In "Controller USB":
    - Change Model to "USB 3"
 12. Click "Add Hardware"
- Add Storage (Disk device) in $HOME/os-images/storage/win10-data.qcow2 with size of 400 GiB
    - Change Bus type to "VirtIO"
    - Change Cache mode to "writeback"
 13. Click "Add Hardware"
- Add Storage (CDROM device) connected to virtio drivers from Red Hat website
    - Change Bus type to "SATA"
    - Change Cache mode to "Hypervisor default"
 14. In "Boot Options":
    - Check "Enable boot menu"
    - Check "VirtIO Disk 1" and "SATA CDROM 1" from the list

I clicked "Begin Installation". It booted successfully to the installation media. I was able to load the VirtIO SCSI driver and see my virtual drives. Then I stopped the installation.

15. Remove "Tablet", "Display Spice", "Sound: ich6", "Serial 1", "Channel spice", "Video QXL", "USB Redirector 1" and "USB Redirector 2"
 16. Click "Add Hardware" and add two "PCI Host Device"s:
    - 0000:01:00:0 ... Tonga PRO...
    - 0000:01:00.1 ... Tonga HDMI Audio...

I clicked the big play-button and changed my monitor input to the dedicated GPU, and it detected no signal. Virtual console said "Graphical console not configured for guest". The system is clearly starting up: it's allocating RAM and using the processor. I tried the same in Fedora 24 earlier this week but it failed in the same stage.

Here's the log file from /var/log/libvirt/qemu/win10.log:
  https://pastebin.com/DSCW0VXP
I don't know if there are some other log files I'd need to include since I'm not too familiar with KVM.

What I've tried:
 - Different Linux distros: Fedora 24 earlier this week, now Arch
 - Different chipsets: i440FX and Q35, with same results
 - pci-stub instead of vfio-pci
 - It's now faulty cable, I'm using it right now
 - Adding `<vendor_id state='on' value='whatever'/>` to `virsh edit`
 - Probably many more things I can't recall right now

I asked the same question over at SuperUser but figured that there'd be more knowledgeable people here.

Thank you,
Elias Kosunen


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]