[vfio-users] Ryzen 1600 pci passthrough degraded performance

[Stefan Maksimovic]

My setup:
Ryzen 1600
Asus B350 Prime Plus
r5 230 (linux host)
gtx 1060 (for passthrough)

The passthrough process itself went without much trouble.
I've been able to boot the VM, the 1060 was properly detected, went on to
install the drivers.
Next, I fired up Dota 2 to take it for a spin and noticed it only gave
around 20 fps.
The 1060 should't have troubles running this game. Haven't run any other
games.

Monitoring the cpu and gpu performance, I noticed that they are not even
remotely loaded.
Searching on the net, I found a thread suggesting adding iommu=pt to the
kernel boot parameters, in addition to
options kvm ignore_msrs=1
options kvm-amd npt=0
to a .conf file in /etc/modprobe.d/, suggesting it may boost performance.

Adding the kernel boot option in addition to the kernel module options
resulted in the VM being unable to boot,
dumping register values with a bunch of other info upon crashing.

By eliminating, I've come to the conclusion that the npt=0 option is the
culprit.
Without it, I am able to boot fine, but still experience the degraded VM
performance in aforementioned game.

I think I've managed to boot the VM with the npt=0 option present once or
twice, in which case the VM reported the cpu having
only one core, despite my setting the -smp in qemu command line to a
specific option.

Information that may prove relevant:
Iommu group in which my card resides, passed only the card itself along
with the audio device, omitting the other two devices
in the ids parameter of vfio-pci:
IOMMU Group 2 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc.
[AMD] Device [1022:1452]
IOMMU Group 2 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
Device [1022:1453]
IOMMU Group 2 22:00.0 VGA compatible controller [0300]: NVIDIA Corporation
GP106 [GeForce GTX 1060 3GB] [10de:1c02] (rev a1)
IOMMU Group 2 22:00.1 Audio device [0403]: NVIDIA Corporation Device
[10de:10f1] (rev a1)

Qemu command line:
qemu-system-x86_64 \
        -enable-kvm \
        -machine type=q35,accel=kvm \
        -m 4096 \
        -mem-path /dev/hugepages \
        -mem-prealloc \
        -cpu host,kvm=off \
        -smp sockets=2,cores=3,threads=2 \
        -usb -usbdevice host:04d9:a09f -usbdevice host:258a:1006 \
        -device vfio-pci,host=22:00.0,multifunction=on,x-vga=on \
        -device vfio-pci,host=22:00.1 \
        -drive
if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/ovmf_code_x64.bin \
        -drive if=pflash,format=raw,file=/tmp/ovmf_vars.bin \
        -boot order=dc \
        -device virtio-scsi-pci,id=scsi \
        -drive file=/vm/win10.img,id=disk,if=none,format=qcow2 -device
scsi-hd,drive=disk \
        -drive
file=/home/stefan/Downloads/virtio-win-0.1.137.iso,id=virtiocd,if=none,format=raw
-device ide-cd,bus=ide.1,drive=virtiocd \
        -netdev user,id=vmnic -device virtio-net,netdev=vmnic \
        -vga none \
        -nographic

I think I've only tried fiddling with the -machine type and x-vga setting,
but that didn't yield any difference.

Finally, my question would be whether anyone has experienced something
remotely similar to what I have?
Any suggestions are welcome, at the moment I don't have any ideas what to
try next.

Thanks in advance,
Stefan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20170516/6d29b57d/attachment.htm>