[vfio-users] Static and stutter on USB devices.

Alex Williamson alex.williamson at redhat.com
Mon May 9 04:34:59 UTC 2016


On Sun, 8 May 2016 19:22:43 -0700
"Lance R. Vick" <lance at lrvick.net> wrote:

> I have some persistent issues with a couple USB devices via vfio/qemu that
> I do not experience when booted native:
> 
> Oculus Rift CV1
> - head-tracking stutter (any game or demo)
> - host CPU ~100% on every core (any game or demo)
>   - at same time Guest CPU may only show ~20% usage
> - audio static from built-in USB DAC
> 
> FiiO K1 USB Headphone DAC/AMP
> - mild but noticeable cuts/jitter
> 
> This may also impact other USB devices in ways I have not yet noticed. Any
> kind of test to quantify this would be helpful. Guest OS was installed on a
> dedicated disk so I can easily native boot to compare tests when needed.
> 
> 3DMark score is only 0.64% lower in VM which indicates to me at least
> things are -mostly- there:
> http://www.3dmark.com/compare/fs/8402818/fs/8403766
> 
> Guest OS is Windows 10 Professional.
> 
> Any advice/debugging tips would be most welcome.
> 
> Here are all the other relevant details I can think of for my setup:

For starters, turn down the kvm module parameter halt_poll_ns to
something like 400000 to see if that brings the host CPU usage down.

echo 400000 > /sys/module/kvm/parameters/halt_poll_ns

Kernel v4.4.8 already includes a change to make this the default.

Next, you have a quad-core+threads processor and you're creating a
quad-core+threads VM; where's the hypervisor supposed to run?  Of
course you're going to get stutters when the guest vCPU is preempted by
the hypervisor or another system process.  Be less aggressive in
allocating vCPUs, virtualization isn't free.  Try a 3-core+threads VM
for example.

Pinning vCPUs can also reduce stutter, it helps to avoid cold caches
and avoids resource conflicts that can occur with threads.  Switch to
libvirt, it makes this easy.

Pinning the emulator can help to make sure it doesn't run on a
conflicting CPU to a vCPU.  Libvirt makes this easy to manage.

Tune irqbalanced to avoid placing interrupts on CPUs used by vCPUs.

Finally, if you're willing to dedicate CPUs completely to the VM, use
the isolcpus= and nohz_full= boot options to avoid the scheduler
running anything on the physical CPUs except for the vCPUs.
> 
> > sudo qemu-system-x86_64 \  
> -enable-kvm \
> -machine type=q35,accel=kvm \
> -cpu host,kvm=off \

Enable hyper-v extensions, but specify the hv_vendor_id option to avoid
Code43 issues.  Some workloads can have huge benefits from this, things
like 3dmark are mostly unaffected.

> -smp 8,sockets=1,cores=4,threads=2 \
> -m 8G \
> -mem-path /dev/hugepages \
> -device virtio-scsi-pci,id=scsi \
> -drive file=/dev/sda,id=disk,format=raw,if=none \
> -device scsi-hd,drive=disk \
> -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on \

x-vga=on isn't needed when you're booting the VM with OVMF.
multifunction=on doesn't do anything useful unless you also specify and
addr= option for the devices.

> -device vfio-pci,host=01:00.1 \
> -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
> \
> -drive if=pflash,format=raw,file=/tmp/OVMF_VARS-pure-efi.fd \
> -nographic \
> -vga none \
> -usb \
> -device vfio-pci,host=00:14.0 \
> -rtc clock=host,base=utc
> 
> 




More information about the vfio-users mailing list