[vfio-users] cpu core pinning with multiple cpus
Erik Adler
erik.adler at gmail.com
Fri Sep 11 22:23:39 UTC 2015
Certain games are giving me terrible frame rates on my GTX 970.
Generally these games are not very demanding when using bare metal.
Unigine Valley Benchmark is doing fine at about 87% native speeds.
Same with some other GPU intensive benchmarks. The games that have bad
fps seem to be taxing the cpu heavily and having latency issues in
passthough.
I am not sure that I have paired my cores correctly. Using Alex's CPU
latency script I get the following. Since this is a dual CPU system I
need to keep everything on the same NUMA node.
There is a definitive pattern but I am not 100% sure that I see the
correctional with lstopo.
| 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
--+------------------------------------------------------------------------
0| 10 8 7 8 7 8 4 4 4 4 4 4 6 7 7 7 7 7 4 4 4 4 4 4
1| 9 10 8 8 8 8 4 4 4 4 4 4 7 7 8 8 8 8 4 4 4 4 4 4
2| 8 8 10 8 8 7 4 4 4 4 5 4 7 8 7 8 8 8 4 4 4 4 4 4
3| 8 8 8 10 8 8 4 5 4 4 4 4 8 7 8 7 8 8 4 4 4 4 4 4
4| 8 8 7 8 10 7 4 4 4 4 4 4 7 8 8 8 7 8 3 4 4 4 4 4
5| 8 8 8 8 8 10 4 4 4 4 4 4 7 8 8 7 8 7 4 4 4 4 4 4
6| 4 4 4 4 4 4 10 6 6 6 6 6 4 4 3 4 4 4 5 7 6 6 6 6
7| 4 4 4 5 4 4 5 10 7 7 7 7 4 4 4 5 5 4 6 7 7 7 7 7
8| 4 5 4 5 4 5 6 7 10 7 7 6 4 4 5 4 4 5 5 6 6 8 8 7
9| 5 5 5 4 4 4 6 7 8 10 8 8 5 4 5 5 5 4 6 8 7 7 8 8
10| 4 4 4 4 4 3 5 6 6 6 10 6 4 4 4 4 4 4 5 6 6 6 5 6
11| 3 3 4 3 4 4 5 5 6 6 6 10 3 3 4 4 4 4 5 6 6 6 6 5
12| 7 8 8 8 8 7 4 5 4 4 4 4 10 8 8 8 8 8 4 5 4 4 5 5
13| 8 7 8 7 7 7 3 4 4 4 4 4 8 10 8 8 7 7 4 4 4 4 4 4
14| 8 8 7 8 8 8 4 4 4 4 4 4 7 8 10 8 7 8 4 5 4 4 4 4
15| 8 7 8 6 8 8 4 4 4 4 4 4 7 7 8 10 8 7 4 5 4 4 4 4
16| 9 8 9 9 8 9 4 5 5 4 5 5 8 8 9 9 10 9 4 5 5 4 5 5
17| 8 7 8 8 8 7 4 4 4 4 4 4 7 8 8 8 7 10 4 4 4 4 4 4
18| 4 4 4 4 4 4 5 7 6 7 7 7 4 4 4 4 4 4 10 6 7 6 7 7
19| 5 5 4 4 4 4 6 6 7 7 7 6 4 4 5 4 4 4 6 10 8 7 7 7
20| 4 5 4 4 4 4 6 8 7 8 8 8 4 4 5 3 4 4 6 8 10 8 8 8
21| 4 4 4 4 4 4 6 5 6 5 6 6 4 4 4 4 4 4 5 6 6 10 7 6
22| 5 4 4 5 4 5 6 7 7 7 7 8 4 5 5 5 5 5 6 8 8 8 10 8
23| 4 4 4 4 4 4 6 7 7 6 7 6 4 4 4 4 4 4 5 7 6 7 7 10
https://i.imgur.com/PQvT2oR.png
Looking at lstopo (url) I have hopefully mapped out my hardware
correctly. In numa node “0” I can see my GTX 970 on PCI 10de:13c2 .
http://i.imgur.com/GBczQvi.png
I am a assuming that if I want to use HT on CPU1 my xml file should
look like this? Have I done something wrong with how I have pinned
cores?
<domain type='kvm'>
<name>Windows</name>
<uuid>cc52dc82-ce9a-45ff-99e6-a92ab0f42b59</uuid>
<memory unit='KiB'>16777216</memory>
<currentMemory unit='KiB'>16777216</currentMemory>
<vcpu placement='static'>8</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='2'/>
<vcpupin vcpu='1' cpuset='3'/>
<vcpupin vcpu='2' cpuset='4'/>
<vcpupin vcpu='3' cpuset='5'/>
<vcpupin vcpu='4' cpuset='14'/>
<vcpupin vcpu='5' cpuset='15'/>
<vcpupin vcpu='6' cpuset='16'/>
<vcpupin vcpu='7' cpuset='17'/>
</cputune>
<os>
<type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
<loader type='rom'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
</os>
<features>
<acpi/>
<apic/>
<pae/>
<kvm>
<hidden state='on'/>
</kvm>
<vmport state='off'/>
</features>
<cpu mode='host-passthrough'>
<topology sockets='1' cores='4' threads='2'/>
</cpu>
<clock offset='localtime'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='yes'/>
<suspend-to-disk enabled='yes'/>
</pm>
<devices>
<emulator>/usr/sbin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/Windows.raw'/>
<target dev='vda' bus='virtio'/>
<boot order='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06'
function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x1'/>
</controller>
<interface type='network'>
<mac address='52:54:00:07:81:15'/>
<source network='default'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
</interface>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='usb' managed='yes'>
<source>
<vendor id='0x0518'/>
<product id='0x0001'/>
</source>
</hostdev>
<hostdev mode='subsystem' type='usb' managed='yes'>
<source>
<vendor id='0x046d'/>
<product id='0xc24a'/>
</source>
</hostdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08'
function='0x0'/>
</memballoon>
</devices>
</domain>
I would like to add that I have changed the Windows 10 guest register
to use MSI for the GPU. That cleared up the audio issues I had when
the GPU was in pw.
Any advice on my xml would be much appreciated.
Is there anyway to "debug" if this is a result of the removed Hyper-V
enlightenments such as ;
<timer name='hypervclock' present='yes'/>
All the best
onryo
PS for formatting I am asking here too
https://virtualkvm.com/viewtopic.php?pid=1000#p1000
More information about the vfio-users
mailing list