[vfio-users] Initialization problem with pass-through video card
Teddy Reed
teddy.reed at gmail.com
Fri May 6 18:44:27 UTC 2016
Hi, Sheldon, I'm seeing almost the identical messages. I have a vastly
different setup and configuration, but I do have an integrated i915,
that seems like the least common denominator. I have tried lots of
hardware and boot configurations, always the same errors.
If I boot ESXi 6.0 and passthrough everything is perfect. So I'm
definitely messing up a Linux/vfio/kvm/qemu option.
Here's the beginning of the logs:
vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19 at 0x270
vfio_ecap_init: 0000:02:00.0 hiding ecap 0x1b at 0x2d0
kvm: zapping shadow pages for mmio generation wraparound
kvm: zapping shadow pages for mmio generation wraparound
kvm [3359]: vcpu3 unhandled rdmsr: 0x641
DMAR: DRHD: handling fault status reg 3
DMAR: DMAR:[DMA Write] Request device [02:00.0] fault addr 28fec0000
DMAR:[fault reason 05] PTE Write access is not set
DMAR: DRHD: handling fault status reg 3
DMAR: DMAR:[DMA Write] Request device [02:00.0] fault addr 28fec3000
DMAR:[fault reason 05] PTE Write access is not set
DMAR: DRHD: handling fault status reg 3
DMAR: DMAR:[DMA Write] Request device [02:00.0] fault addr 28fec4000
DMAR:[fault reason 05] PTE Write access is not set
DMAR: DRHD: handling fault status reg 3
DMAR: DMAR:[DMA Write] Request device [02:00.0] fault addr 28fec5000
DMAR:[fault reason 05] PTE Write access is not set
On Fri, May 6, 2016 at 11:07 AM, Sheldon Palmer <slf_palmer at hotmail.com> wrote:
> Ok so even though I have gone through Alex’s “step by step”
> (http://vfio.blogspot.com/) several times over, I think I am still missing
> something because each time I get the same result. That is: my pass-through
> video card doesn’t get initialized when I actually add it and boot the VM.
> There is just a quick flicker then a black screen. In fact, I don’t even
> think the OS boots because I can’t connect to it via VNC nor TeamViewer. The
> VM boots okay if I remove the video card.
>
>
Same here!
>
> Specs:
>
> Processor: Intel Core i7-3770 Quad Core
> Host: Intel onboard video card with i915 driver / Gentoo
> Pass-through: ASUS GeForce GTX 980Ti STRIX / Windows 8.1 Pro
> qemu-2.5.1 / libvirt-1.3.1 / virt-manager-1.3.2 / kernel: linux-4.1.15
>
I'm using an i7-6700K
ASUS Z170-A with the i915 and Ubuntu 14.04
Passing an ATI ASUS 390X / Windows 10 (with all updates)
qemu built from master, linux-4.1 and 4.2 and 4.4
>
>
> Now I think I am a bit confused as it relates to what is actually needed
> these days for a set-up with the i915 driver being used on the host side. If
> I understand correctly, the patch is no longer needed if OVMF is being used
> correct?
>
>
> In addition, I noticed that I’m getting some DMA Write messages but I
> thought adding “vfio_iommu_type1.allow_unsafe_interrupts=1“ to modprobe.d
> would address it. Anyways, what am I overlooking here guys? Thanks is
> advance.
>
I've also tried to add: intel_iommu=on,pt
Also with and without allowing insafe interrupts.
>
> Additional Info:
>
> # cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-4.1.15-gentoo-r1
> root=UUID=b8357358-a71d-4250-83e5-5505e8ad5eba ro linux intel_iommu=on
>
> IOMMU group 1
> 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen
> Core processor PCI Express Root Port [8086:0151] (rev 09)
> 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200
> [GeForce GTX 980 Ti] [10de:17c8] (rev a1)
> 01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fb0] (rev
> a1)
>
> # cat /etc/modprobe.d/vfio.conf
> options vfio-pci ids=8086:0151,10de:17c8,10de:0fb0
> options vfio-pci disable_vga=1
>
> # cat /etc/modprobe.d/kvm.conf
> # if vfio-pci was built as a module ( default on arch & ubuntu )
> options vfio_iommu_type1 allow_unsafe_interrupts=1
>
> # Some applications like Passmark Performance Test and SiSoftware Sandra
> crash the VM without this:
> options kvm ignore_msrs=1
>
> # dmesg | grep -i vfio
> [ 3.895580] VFIO - User Level meta-driver version: 0.3
> [ 3.920197] vfio_pci: add [8086:0151[ffff:ffff]] class 0x000000/00000000
> [ 3.930899] vfio_pci: add [10de:17c8[ffff:ffff]] class 0x000000/00000000
> [ 3.941936] vfio_pci: add [10de:0fb0[ffff:ffff]] class 0x000000/00000000
> [ 422.613903] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e at 0x258
> [ 422.613919] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19 at 0x900
> [ 1953.837834] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1e at 0x258
> [ 1953.837843] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19 at 0x900
>
> # dmesg -T | grep -e DMAR -e IOMMU -e hardware
> [Thu May 5 21:40:04 2016] ACPI: DMAR 0x00000000C8972760 0000B8 (v01 INTEL
> SNB 00000001 INTL 00000001)
> [Thu May 5 21:40:04 2016] Intel-IOMMU: enabled
> [Thu May 5 21:40:04 2016] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap
> c0000020e60262 ecap f0101a
> [Thu May 5 21:40:04 2016] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap
> c9008020660262 ecap f0105a
> [Thu May 5 21:40:04 2016] DMAR: No ATSR found
> [Thu May 5 21:40:04 2016] IOMMU: dmar0 using Queued invalidation
> [Thu May 5 21:40:04 2016] IOMMU: dmar1 using Queued invalidation
> [Thu May 5 21:40:04 2016] IOMMU: Setting RMRR:
> [Thu May 5 21:40:04 2016] IOMMU: Setting identity map for device
> 0000:00:02.0 [0xcb000000 - 0xcf1fffff]
> [Thu May 5 21:40:04 2016] IOMMU: Setting identity map for device
> 0000:00:14.0 [0xc88e5000 - 0xc88f6fff]
> [Thu May 5 21:40:04 2016] IOMMU: Setting identity map for device
> 0000:00:1a.0 [0xc88e5000 - 0xc88f6fff]
> [Thu May 5 21:40:04 2016] IOMMU: Setting identity map for device
> 0000:00:1d.0 [0xc88e5000 - 0xc88f6fff]
> [Thu May 5 21:40:04 2016] IOMMU: Prepare 0-16MiB unity mapping for LPC
> [Thu May 5 21:40:04 2016] IOMMU: Setting identity map for device
> 0000:00:1f.0 [0x0 - 0xffffff]
> [Thu May 5 21:40:04 2016] [drm] DMAR active, disabling use of stolen memory
> [Thu May 5 21:40:05 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:37 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:37 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:37 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:37 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:38 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:38 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:38 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:38 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:38 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:38 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:40 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:40 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
> [Thu May 5 21:40:41 2016] dmar: DMAR:[DMA Write] Request device [00:02.0]
> fault addr 29ff005000
> DMAR:[fault reason 05] PTE Write access is not
> set
>
> MY XML:
>
>
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
> <name>win8.1-pro</name>
> <uuid>39f1e987-667c-43ff-8641-e10f017d370c</uuid>
> <memory unit='KiB'>6291456</memory>
> <currentMemory unit='KiB'>6291456</currentMemory>
> <vcpu placement='static'>4</vcpu>
> <cputune>
> <vcpupin vcpu='0' cpuset='2'/>
> <vcpupin vcpu='1' cpuset='3'/>
> <vcpupin vcpu='2' cpuset='6'/>
> <vcpupin vcpu='3' cpuset='7'/>
> </cputune>
> <os>
> <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
I'm using the q35 but have tried a few of the i440fxs too.
> <loader readonly='yes'
> type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
> <nvram>/var/lib/libvirt/qemu/nvram/win8.1-pro_VARS.fd</nvram>
> <bootmenu enable='yes'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> <hyperv>
> <relaxed state='off'/>
> <vapic state='off'/>
> <spinlocks state='off'/>
> </hyperv>
> <kvm>
> <hidden state='on'/>
> </kvm>
> <vmport state='off'/>
> </features>
> <cpu mode='host-passthrough'>
> <topology sockets='1' cores='2' threads='2'/>
> </cpu>
> <clock offset='localtime'>
> <timer name='rtc' tickpolicy='catchup'/>
> <timer name='pit' tickpolicy='delay'/>
> <timer name='hpet' present='no'/>
> </clock>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>restart</on_crash>
> <pm>
> <suspend-to-mem enabled='no'/>
> <suspend-to-disk enabled='no'/>
> </pm>
> <devices>
> <emulator>/usr/bin/qemu-system-x86_64</emulator>
> <disk type='file' device='disk'>
> <driver name='qemu' type='raw'/>
> <source file='/mnt/new_root/home/sheldon/VMs/win8.1-pro'/>
> <target dev='vda' bus='virtio'/>
> <boot order='1'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
> </disk>
> <controller type='usb' index='0' model='ich9-ehci1'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x7'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci1'>
> <master startport='0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0' multifunction='on'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci2'>
> <master startport='2'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x1'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci3'>
> <master startport='4'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x2'/>
> </controller>
> <controller type='pci' index='0' model='pci-root'/>
> <interface type='direct'>
> <mac address='52:54:00:7e:e3:8a'/>
> <source dev='enp3s0' mode='bridge'/>
> <model type='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
> </interface>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
> </hostdev>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </hostdev>
> <memballoon model='virtio'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
> </memballoon>
> </devices>
> <qemu:commandline>
> <qemu:arg value='-drive'/>
> <qemu:arg
> value='if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd'/>
> </qemu:commandline>
> </domain>
>
Hope my notes help!
--
Teddy Reed V
More information about the vfio-users
mailing list