[vfio-users] Static and stutter on USB devices.
Lance R. Vick
lance at lrvick.net
Mon May 9 02:22:43 UTC 2016
I have some persistent issues with a couple USB devices via vfio/qemu that
I do not experience when booted native:
Oculus Rift CV1
- head-tracking stutter (any game or demo)
- host CPU ~100% on every core (any game or demo)
- at same time Guest CPU may only show ~20% usage
- audio static from built-in USB DAC
FiiO K1 USB Headphone DAC/AMP
- mild but noticeable cuts/jitter
This may also impact other USB devices in ways I have not yet noticed. Any
kind of test to quantify this would be helpful. Guest OS was installed on a
dedicated disk so I can easily native boot to compare tests when needed.
3DMark score is only 0.64% lower in VM which indicates to me at least
things are -mostly- there:
http://www.3dmark.com/compare/fs/8402818/fs/8403766
Guest OS is Windows 10 Professional.
Any advice/debugging tips would be most welcome.
Here are all the other relevant details I can think of for my setup:
> uname -r
4.4.5-1-vfio
> lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 94
Model name: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
Stepping: 3
CPU MHz: 4200.000
CPU max MHz: 4200.0000
CPU min MHz: 800.0000
BogoMIPS: 8019.82
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep
mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht
tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts
rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq
dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid
sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx
f16c rdrand lahf_lm abm 3dnowprefetch ida arat epb pln pts dtherm hwp
hwp_notify hwp_act_window hwp_epp intel_pt tpr_shadow vnmi
flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms
invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1
> sudo lshw -short
H/W path Device Class Description
=================================================================
system System Product Name (SKU)
/0 bus MAXIMUS VIII EXTREME
/0/0 memory 64KiB BIOS
/0/42 memory 128KiB L1 cache
/0/43 memory 128KiB L1 cache
/0/44 memory 1MiB L2 cache
/0/45 memory 8MiB L3 cache
/0/46 processor Core i7 (To Be
Filled By O.E.M.)
/0/47 memory 32GiB System Memory
/0/47/0 memory 16GiB DIMM
Synchronous 2133 MHz (0.5 ns)
/0/47/1 memory [empty]
/0/47/2 memory 16GiB DIMM
Synchronous 2133 MHz (0.5 ns)
/0/47/3 memory [empty]
/0/100 bridge Skylake Host
Bridge/DRAM Registers
/0/100/1 bridge Skylake PCIe
Controller (x16)
/0/100/1/0 display GM200 [GeForce GTX 980 Ti]
/0/100/1/0.1 multimedia NVIDIA Corporation
/0/100/1.1 bridge Skylake PCIe
Controller (x8)
/0/100/1.1/0 display Caicos PRO
[Radeon HD 7450]
/0/100/1.1/0.1 multimedia Caicos HDMI
Audio [Radeon HD 6400 Series]
/0/100/2 display Skylake
Integrated Graphics
/0/100/14 bus Sunrise Point-H
USB 3.0 xHCI Controller
/0/100/16 communication Sunrise Point-H
CSME HECI #1
/0/100/17 storage Sunrise Point-H
SATA controller [AHCI mode]
/0/100/1b bridge Sunrise Point-H
PCI Root Port #17
/0/100/1c bridge Sunrise Point-H
PCI Express Root Port #1
/0/100/1c/0 bus ASM1142 USB 3.1
Host Controller
/0/100/1c/0/0 usb2 bus xHCI Host Controller
/0/100/1c/0/0/1 storage CDDVDW SU-208DB
/0/100/1c/0/1 usb1 bus xHCI Host Controller
/0/100/1c/0/1/2 multimedia HD Pro Webcam C920
/0/100/1c.2 bridge Sunrise Point-H
PCI Express Root Port #3
/0/100/1c.2/0 bridge ASMedia Technology Inc.
/0/100/1c.2/0/1 bridge ASMedia Technology Inc.
/0/100/1c.2/0/1/0 bus VL805 USB 3.0
Host Controller
/0/100/1c.2/0/1/0/0 usb4 bus xHCI Host Controller
/0/100/1c.2/0/1/0/1 usb3 bus xHCI Host Controller
/0/100/1c.2/0/1/0/1/1 bus USB2.0 Hub
/0/100/1c.2/0/1/0/1/1/4 generic FT240X USB FIFO
/0/100/1c.2/0/2 bridge ASMedia Technology Inc.
/0/100/1c.2/0/2/0 network BCM4360 802.11ac
Wireless Network Adapter
/0/100/1c.2/0/3 bridge ASMedia Technology Inc.
/0/100/1c.2/0/3/0 storage ASM1062 Serial
ATA Controller
/0/100/1c.2/0/4 bridge ASMedia Technology Inc.
/0/100/1c.2/0/5 bridge ASMedia Technology Inc.
/0/100/1c.2/0/5/0 bus ASM1042A USB 3.0
Host Controller
/0/100/1c.2/0/5/0/0 usb6 bus xHCI Host Controller
/0/100/1c.2/0/5/0/0/2 bus USB3.0 Hub
/0/100/1c.2/0/5/0/0/2/4 bus USB3.0 Hub
/0/100/1c.2/0/5/0/1 usb5 bus xHCI Host Controller
/0/100/1c.2/0/5/0/1/1 generic USB 10/100 LAN
/0/100/1c.2/0/5/0/1/2 bus USB2.0 Hub
/0/100/1c.2/0/5/0/1/2/1 bus USB2.0 Hub
/0/100/1c.2/0/5/0/1/2/4 bus USB2.0 Hub
/0/100/1c.2/0/5/0/1/2/4/4 input Yubikey 4 OTP+U2F+CCID
/0/100/1c.2/0/6 bridge ASMedia Technology Inc.
/0/100/1c.2/0/7 bridge ASMedia Technology Inc.
/0/100/1c.4 bridge Sunrise Point-H
PCI Express Root Port #5
/0/100/1c.4/0 bridge DSL6540
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/0 bridge DSL6540
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/1 bridge DSL6540
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/2 bridge DSL6540
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/2/0 bus DSL6540 USB 3.1
Controller [Alpine Ridge]
/0/100/1c.4/0/2/0/0 usb8 bus xHCI Host Controller
/0/100/1c.4/0/2/0/1 usb7 bus xHCI Host Controller
/0/100/1c.4/0/2/0/1/2 bus USB 2.0 Hub
/0/100/1c.4/0/2/0/1/2/2 input CST Laser Trackball
/0/100/1c.4/0/2/0/1/2/3 input ErgoDox ergonomic keyboard
/0/100/1c.4/0/4 bridge DSL6540
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015]
/0/100/1d bridge Sunrise Point-H
PCI Express Root Port #9
/0/100/1d/0 storage NVMe SSD Controller
/0/100/1f bridge Sunrise Point-H
LPC Controller
/0/100/1f.2 memory Memory controller
/0/100/1f.3 multimedia Sunrise Point-H HD Audio
/0/100/1f.4 bus Sunrise Point-H SMBus
/0/100/1f.6 enp0s31f6 network Ethernet
Connection (2) I219-V
/1 power To Be Filled By O.E.M.
/2 scsi8 storage
/3 virbr0-nic network Ethernet interface
/4 enp11s0u1 network Ethernet interface
> cat /proc/cmdline
initrd=\initramfs-linux-vfio.img
cryptdevice=UUID=bc838115-107d-41ef-a076-6a59a2a6bb43:cryptroot
root=/dev/mapper/cryptroot pcie_acs_override=downstream intel_iommu=on
rw
> dmesg | grep -e DMAR -e IOMMU
[ 0.000000] Warning: PCIe ACS overrides enabled; This may allow
non-IOMMU protected peer-to-peer DMA
[ 0.000000] ACPI: DMAR 0x00000000365D85A8 0000B8 (v01 INTEL SKL
00000001 INTL 00000001)
[ 0.000000] DMAR: IOMMU enabled
[ 0.076983] DMAR: Host address width 39
[ 0.076984] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.076989] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap
1c0000c40660462 ecap 7e3ff0505e
[ 0.076990] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.076993] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap
d2008c40660462 ecap f050da
[ 0.076994] DMAR: RMRR base: 0x0000003630a000 end: 0x00000036329fff
[ 0.076994] DMAR: RMRR base: 0x00000038800000 end: 0x0000003affffff
[ 0.076996] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.076996] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.076997] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[ 0.076997] DMAR-IR: Use 'intremap=no_x2apic_optout' to override
the BIOS setting.
[ 0.078318] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 0.587024] DMAR: [Firmware Bug]: RMRR entry for device 04:00.0 is
broken - applying workaround
[ 0.587027] DMAR: [Firmware Bug]: RMRR entry for device 0b:00.0 is
broken - applying workaround
[ 0.587030] DMAR: No ATSR found
[ 0.587086] DMAR: dmar0: Using Queued invalidation
[ 0.587296] DMAR: dmar1: Using Queued invalidation
[ 0.587301] DMAR: Setting RMRR:
[ 0.587327] DMAR: Setting identity map for device 0000:00:02.0
[0x38800000 - 0x3affffff]
[ 0.587335] DMAR: Setting identity map for device 0000:00:14.0
[0x3630a000 - 0x36329fff]
[ 0.587343] DMAR: Setting identity map for device 0000:04:00.0
[0x3630a000 - 0x36329fff]
[ 0.587350] DMAR: Setting identity map for device 0000:0b:00.0
[0x3630a000 - 0x36329fff]
[ 0.587353] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.587357] DMAR: Setting identity map for device 0000:00:1f.0 [0x0
- 0xffffff]
[ 0.587383] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 108.828150] AMD IOMMUv2 driver by Joerg Roedel <jroedel at suse.de>
[ 108.828152] AMD IOMMUv2 functionality not available on this system
> for id in 00:14.0 01:00.0 01:00.1; do sudo lspci -nn -v -s $id; done
00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB
3.0 xHCI Controller [8086:a12f] (rev 31) (prog-if 30 [XHCI])
Subsystem: ASUSTeK Computer Inc. Device [1043:8694]
Flags: bus master, medium devsel, latency 0, IRQ 171
Memory at 2fff010000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [70] Power Management version 2
Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
Kernel driver in use: vfio-pci
Kernel modules: xhci_pci
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200
[GeForce GTX 980 Ti] [10de:17c8] (rev a1) (prog-if 00 [VGA
controller])
Subsystem: eVga.com. Corp. Device [3842:4995]
Flags: bus master, fast devsel, latency 0, IRQ 170
Memory at db000000 (32-bit, non-prefetchable) [size=16M]
Memory at 90000000 (64-bit, prefetchable) [size=256M]
Memory at a0000000 (64-bit, prefetchable) [size=32M]
I/O ports at e000 [size=128]
Expansion ROM at dc000000 [disabled] [size=512K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Legacy Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [258] L1 PM Substates
Capabilities: [128] Power Budgeting <?>
Capabilities: [420] Advanced Error Reporting
Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: [900] #19
Kernel driver in use: vfio-pci
Kernel modules: nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fb0] (rev a1)
Subsystem: eVga.com. Corp. Device [3842:4995]
Flags: bus master, fast devsel, latency 0, IRQ 17
Memory at dc080000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
> sudo sysctl vm.nr_hugepages=7168
vm.nr_hugepages = 7168
> sudo qemu-system-x86_64 \
-enable-kvm \
-machine type=q35,accel=kvm \
-cpu host,kvm=off \
-smp 8,sockets=1,cores=4,threads=2 \
-m 8G \
-mem-path /dev/hugepages \
-device virtio-scsi-pci,id=scsi \
-drive file=/dev/sda,id=disk,format=raw,if=none \
-device scsi-hd,drive=disk \
-device vfio-pci,host=01:00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
\
-drive if=pflash,format=raw,file=/tmp/OVMF_VARS-pure-efi.fd \
-nographic \
-vga none \
-usb \
-device vfio-pci,host=00:14.0 \
-rtc clock=host,base=utc
--
Lance R. Vick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160508/7e99d600/attachment.htm>
More information about the vfio-users
mailing list