[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [vfio-users] W10: installing RX Vega 56 drivers locks up VM



Hi,

If you using Radeon RX with i440fx type vm, GPU attached to pcie-root-complex trick is maybe needed.

i440fx and recent AMD Crimson drivers

Remove your Radeon RX Vega 56 passthrough config from <hostdev>~~</hostdev>section and using libvirt "<qemu:arg>" option, some thing like this.
 <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,rombar=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.1,bus=root.1,addr=00.1'/>
  </qemu:commandline>

Or, Use Q35 type VM with Intel ioh3420 pcie-root-port. For example, see below.

    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>

~~~~~ snip ~~~~~~

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0xaf' slot='0x00' function='0x0'/>
      </source>
      <rom bar='on'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0xaf' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
    </hostdev>

"model name=ioh3420" is Intel emulated pcie-root-port.
defalut "model name=pcie-root-port" is QEMU Paravirtualized pcie-root-port.
Radeon driver with "ioh3420" root port trick works fine on my workstation.
(but... not RX Vega 56, I'm using RX 460.)

Cheers,
Hisato.


> On Thu, 28 Mar 2019 at 20:32, Tmplt <tmplt dragons rocks> wrote:
> >
> > On 3/28/19 8:02 PM, Kash Pande wrote:
> > > kvm module parameter ignore_msrs is needed here.
> > Is this much different from the
> >
> >   echo 1 > /sys/module/kvm/parameters/ignore_msrs
>
> I have a very similar setup with a Sapphire RX Vega 56 and also
> observing this lockup.
> I am using an unpatched Debian 4.19.0. Qemu 3.1.0. Libvirt 5.0.0. A
> single host core is pegged at 100%. I've attached my XML too.
> $ cat /sys/module/kvm/parameters/ignore_msrs
> Y
> I cannot find any logs indicating any problems...
> $ sudo journalctl | grep kvm
> Mar 28 20:35:52 socrates kernel: Command line:
> BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64
> root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt
> amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB
> hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1
> kvm.ignore_msrs=1
> Mar 28 20:35:52 socrates kernel: Kernel command line:
> BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64
> root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt
> amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB
> hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1
> kvm.ignore_msrs=1
> Mar 28 20:35:52 socrates kernel: kvm: Nested Virtualization enabled
> Mar 28 20:35:52 socrates kernel: kvm: Nested Paging enabled
> $ sudo journalctl | grep msrs
> Mar 28 20:35:52 socrates kernel: Command line:
> BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64
> root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt
> amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB
> hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1
> kvm.ignore_msrs=1
> Mar 28 20:35:52 socrates kernel: Kernel command line:
> BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64
> root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt
> amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB
> hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1
> kvm.ignore_msrs=1
>
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
>   <name>win10_backup</name>
>   <uuid>4b697e59-793e-450f-8722-38e785831796</uuid>
>   <title>Windows 10</title>
>   <description>Windows gaming VM</description>
>   <memory unit='KiB'>16777216</memory>
>   <currentMemory unit='KiB'>16777216</currentMemory>
>   <vcpu placement='static'>8</vcpu>
>   <iothreads>1</iothreads>
>   <cputune>
>     <vcpupin vcpu='0' cpuset='8'/>
>     <vcpupin vcpu='1' cpuset='9'/>
>     <vcpupin vcpu='2' cpuset='10'/>
>     <vcpupin vcpu='3' cpuset='11'/>
>     <vcpupin vcpu='4' cpuset='12'/>
>     <vcpupin vcpu='5' cpuset='13'/>
>     <vcpupin vcpu='6' cpuset='14'/>
>     <vcpupin vcpu='7' cpuset='15'/>
>     <emulatorpin cpuset='0-1'/>
>     <iothreadpin iothread='1' cpuset='0-1'/>
>   </cputune>
>   <os>
>     <type arch='x86_64' machine='pc-i440fx-3.1'>hvm</type>
>     <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>
>     <nvram>/var/lib/libvirt/qemu/nvram/win10_backup_VARS.fd</nvram>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>     <hyperv>
>       <relaxed state='on'/>
>       <vapic state='on'/>
>       <spinlocks state='on' retries='8191'/>
>     </hyperv>
>     <vmport state='off'/>
>   </features>
>   <cpu mode='host-passthrough' check='none'>
>     <topology sockets='1' cores='4' threads='2'/>
>     <cache level='3' mode='emulate'/>
>     <feature policy='require' name='topoext'/>
>   </cpu>
>   <clock offset='localtime'>
>     <timer name='rtc' tickpolicy='catchup'/>
>     <timer name='pit' tickpolicy='delay'/>
>     <timer name='hpet' present='no'/>
>     <timer name='hypervclock' present='yes'/>
>   </clock>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>destroy</on_crash>
>   <pm>
>     <suspend-to-mem enabled='no'/>
>     <suspend-to-disk enabled='no'/>
>   </pm>
>   <devices>
>     <emulator>/usr/bin/kvm</emulator>
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='raw' io='threads'/>
>       <source file='/var/lib/libvirt/images/win10-clean_backup.raw'/>
>       <target dev='vda' bus='virtio'/>
>       <boot order='2'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
>     </disk>
>     <disk type='file' device='cdrom'>
>       <driver name='qemu' type='raw'/>
>       <source file='/var/lib/libvirt/images/virtio-win-0.1.160.iso'/>
>       <target dev='sda' bus='sata'/>
>       <readonly/>
>       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>     </disk>
>     <controller type='pci' index='0' model='pci-root'/>
>     <controller type='usb' index='0' model='nec-xhci'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
>     </controller>
>     <controller type='sata' index='0'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
>     </controller>
>     <interface type='network'>
>       <mac address='52:54:00:39:d1:fc'/>
>       <source network='default'/>
>       <model type='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
>     </interface>
>     <input type='mouse' bus='ps2'/>
>     <input type='keyboard' bus='ps2'/>
>     <sound model='ich6'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
>     </sound>
>     <hostdev mode='subsystem' type='pci' managed='yes'>
>       <source>
>         <address domain='0x0000' bus='0x13' slot='0x00' function='0x0'/>
>       </source>
>       <rom bar='off'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
>     </hostdev>
>     <hostdev mode='subsystem' type='usb' managed='yes'>
>       <source>
>         <vendor id='0x046d'/>
>         <product id='0xc24c'/>
>         <address bus='3' device='7'/>
>       </source>
>       <address type='usb' bus='0' port='1'/>
>     </hostdev>
>     <hostdev mode='subsystem' type='usb' managed='yes'>
>       <source>
>         <vendor id='0x046d'/>
>         <product id='0xc228'/>
>         <address bus='3' device='9'/>
>       </source>
>       <address type='usb' bus='0' port='2'/>
>     </hostdev>
>     <memballoon model='virtio'>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
>     </memballoon>
>   </devices>
>   <qemu:commandline>
>     <qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
>     <qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/>
>   </qemu:commandline>
> </domain>


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]