[vfio-users] BIOS memory mapping or chipset issue with MMIO prevents IGD pass-through in legacy mode?

[globalgorrilla at fastmail.fm]

First, thank you  for your blog posts and your work on vfio and in this
area. 

On Wed, Jul 27, 2016, at 05:20 PM, Alex Williamson wrote:
> On Wed, 27 Jul 2016 17:01:21 -0700
> globalgorrilla at fastmail.fm wrote:
> 
> > Hello,
> > 
> > Do you reckon this is what some bugzilla's find to be a motherboard
> > memory mapping issue or a chipset issue? 
> > 
> > Any ideas on working around it besides opting the IGD out of MMIO? 
> > 
> > Doing that prevents it from being used with vfio-pci and qemu.
> > 
> > ---
> > 
> > i7-4790k, ASUS Z97-WS (BIOS 2403), kernel 4.7, qemu v2.7.0-rc0
> > 
> > % lspci -vvn -s 00:02
> > 00:02.0 0300: 8086:0412 (rev 06) (prog-if 00 [VGA controller])
> >         Subsystem: 1043:8534
> >         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> >         ParErr- Stepping- SERR- FastB2B- DisINTx-
> >         Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
> >         <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Latency: 0
> >         Interrupt: pin A routed to IRQ 11
> >         Region 0: Memory at df800000 (64-bit, non-prefetchable)
> >         [size=4M]
> >         Region 2: Memory at b0000000 (64-bit, prefetchable) [size=256M]
> >         Region 4: I/O ports at f000 [size=64]
> >         [virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
> >         Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit-
> >                 Address: 00000000  Data: 0000
> >         Capabilities: [d0] Power Management version 2
> >                 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
> >                 PME(D0-,D1-,D2-,D3hot-,D3cold-)
> >                 Status: D3 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> >         Capabilities: [a4] PCI Advanced Features
> >                 AFCap: TP+ FLR+
> >                 AFCtrl: FLR-
> >                 AFStatus: TP-
> >         Kernel driver in use: vfio-pci
> >         Kernel modules: i915
> > 
> > % cat /proc/cmdline
> > ... intel_iommu=on iommu=pt default_hugepagesz=1G hugepagesz=1G
> > hugepages=16 kvm.ignore_msrs=1 intremap=no_x2apic_optout console=ttyS0
> > video=vesafb:off,efifb:off ...
> > 
> > % cat /etc/modprobe.d/blacklist.conf 
> > blacklist efifb
> > install efifb /bin/false
> > 
> > % cat /etc/modprobe.d/vfio.conf 
> > options vfio-pci ids=8086:0412
> > ---
> > 
> > First  I re-built the kernel with efifb as a module for the modprobe
> > blacklist to prevent efifb from binding.  Something in systemd-boot?
> 
> Not necessary, the video=efifb:off should take care of it.

That's what I thought as well. Perhaps I missed something  but after I
built it as a module it no longer binds.

>  
> > Now enabling CSM in the BIOS has made the Video ROM available (whilst
> > still booting with UEFI) but qemu still can't enable VGA access and
> > legacy pass-through mode is disabled.
> > 
> > ---
> > 
> > % grep Video /proc/iomem        
> > 00000000-00000000 : Video ROM
> 
> Recent kernels won't show /proc/iomem or /proc/ioport to regular users
> as a security measure.  You need sudo to see anything.

Noted. 

Should I be looking for anything else here? I am able to see that Video
ROM line now after enabling CSM.

> 
> > % /usr/bin/qemu-system-x86_64 -name guest=win10_1 -S \
> > -machine pc-i440fx-2.6,accel=kvm,usb=off \
> > -cpu host -m 8192 \
> > -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu \
> > -smp 6,sockets=1,cores=3,threads=2 \
> > -nographic -no-user-config -nodefaults \
> > -device vfio-pci,host=00:02.0,id=hostdev0,bus=pci.0,addr=0x2 \
> > -msg timestamp=on \
> > -bios /usr/share/qemu/bios.bin
> > 
> > ...
> > [   45.426546] DMAR: DRHD: handling fault status reg 3
> > [   45.484901] DMAR: [DMA Read] Request device [00:02.0] fault addr
> > 8f2c0000 [fault reason 06] PTE Read access is not set
> > [   45.612929] DMAR: DRHD: handling fault status reg 2
> > [   45.671267] DMAR: [DMA Read] Request device [00:02.0] fault addr
> > 8f314000 [fault reason 06] PTE Read access is not set
> > [   45.799288] DMAR: DRHD: handling fault status reg 2
> > [   45.857634] DMAR: [DMA Read] Request device [00:02.0] fault addr
> > 8f3a7000 [fault reason 06] PTE Read access is not set
> > 
> > qemu.log:
> > qemu-system-x86_64: -device
> > vfio-pci,host=00:02.0,id=hostdev0,bus=pci.0,addr=0x2: IGD device
> > 0000:00:02.0 failed to enable VGA access, legacy mode disabled
> 
> Is you kernel built with CONFIG_VFIO_PCI_VGA=y?

Yes.

% zcat /proc/config.gz | grep CONFIG_VFIO_PCI_VGA
CONFIG_VFIO_PCI_VGA=y