[vfio-users] Finding a suitable replacement mainboard for GPU passthrough

[Alex Williamson]

On Wed, Sep 28, 2016 at 1:29 PM, Martin Schrodt <martin at schrodt.org> wrote:

> Dear community,
>
> after buying a new rig a week ago, I noticed several problems with the
> mainboard I chose, so I am going to send it back. It's an ASRock Z 170
> Extreme 4.
>
> Problems I have:
>
> 1) The board does not like the RAM I ordered. I have a kit DDR4-3200
> DIMM CL14-14-14-34 from G-Skill that will crap out when running at it's
> specified speed, right now it's stable at 2800.
>
> 2) It won't suspend correctly if I leave it's USB-3.1 controller to
> Linux (will wake up immediately, complaining that the USB controller
> prevented proper suspend) (ASMedia Technology Inc. ASM1142)
>

A poorly behaving ASMedia device, shocking </sarcasm>

3) The first and the second PCIe-slot are in the same IOMMU-group,
> preventing me to use a second GPU for the host in slot 2 without
> ACS-override. (*1)
>

Of course, you need a certain class of processors to have ACS, Z170 doesn't
support any of those processors.  See
http://vfio.blogspot.com/2015/10/intel-processors-with-acs-support.html
 You need an X99 motherboard for that.


> 4) When resuming from suspend, the BIOS (I presume, someone with another
> board from ASRock has the same problems) fucks up TSC, which causes the
> kernel to turn it off, and the VM to stutter badly afterwards. I have to
> reboot to get a performing VM. (*2)
>
> 5) The VM is dead after I suspend/resume the host.
>

Hmm, if you're thinking a VM with assigned devices should survive a host
suspend/resume, I'm not sure that's a reasonable expectation right now.


> I need help in finding a good board quickly, so my downtime won't be too
> much. So I reach out to y'all. Is someone out there who can suggest a
> mainboard that does provide remedy for the points above?
>
> I am very grateful for your help!
>
> (*1) script to list IOMMU group with contained device:
>
> for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1
> -mindepth 1 -type d); do echo "IOMMU group $(basename "$iommu_group")";
> for device in $(\ls -1 "$iommu_group"/devices/); do echo -n $'\t'; lspci
> -nns "$device"; done; done
>
> (*2) To check whether tsc is enabled:
>
> cat /sys/devices/system/clocksource/clocksource0/available_clocksource
>
> The output should contain "tsc" before and after suspend to ram.
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160928/66562274/attachment.htm>