[vfio-users] How to split iommu group on [xeon e3-1220 v5 + supermicro x11ssh-ln4f]

[Alex Williamson]

On Tue, Apr 5, 2016 at 3:13 AM, Tran The Anh <anh.tranthe at thegioididong.com>
wrote:

> Hi,
>
> I would like to passthrough some intel i210 ports to KVM's VM, but has
> encountered the iommu group issue:
>
> /sys/kernel/iommu_groups/0/devices/0000:00:00.0
> /sys/kernel/iommu_groups/1/devices/0000:00:14.0 <= usb
> /sys/kernel/iommu_groups/1/devices/0000:00:14.2
> /sys/kernel/iommu_groups/2/devices/0000:00:16.0
> /sys/kernel/iommu_groups/2/devices/0000:00:16.1
> /sys/kernel/iommu_groups/3/devices/0000:00:17.0 <= sata
> /sys/kernel/iommu_groups/4/devices/0000:00:1c.0 <= pch port
> /sys/kernel/iommu_groups/4/devices/0000:00:1c.1 <= pch port
> /sys/kernel/iommu_groups/4/devices/0000:00:1c.2 <= pch port
> /sys/kernel/iommu_groups/4/devices/0000:00:1c.3 <= pch port
> /sys/kernel/iommu_groups/4/devices/0000:00:1c.6 <= pch port
> /sys/kernel/iommu_groups/4/devices/0000:01:00.0 <= i210
> /sys/kernel/iommu_groups/4/devices/0000:02:00.0 <= i210
> /sys/kernel/iommu_groups/4/devices/0000:03:00.0 <= i210
> /sys/kernel/iommu_groups/4/devices/0000:04:00.0 <= i210
> /sys/kernel/iommu_groups/4/devices/0000:05:00.0
> /sys/kernel/iommu_groups/4/devices/0000:06:00.0 <= vga aspeed 2400
> /sys/kernel/iommu_groups/5/devices/0000:00:1f.0
> /sys/kernel/iommu_groups/5/devices/0000:00:1f.2
> /sys/kernel/iommu_groups/5/devices/0000:00:1f.4
>
>
> My system is:
>
> Intel Xeon e3-1220 v5
> Supermicro x11ssh-ln4f
>
>
> According to board's diagram:
> http://www.supermicro.com/manuals/motherboard/C236/MNL-1778.pdf
> Those i210s are connected to PCH.
>
> It seems that PCH ports have ACS capabilities:
>
> lspci -vvvnk | egrep -i 'Access|Rev'
> 00:17.0 0106: 8086:a102 (rev 31) (prog-if 01 [AHCI 1.0])
> 00:1c.0 0604: 8086:a110 (rev f1) (prog-if 00 [Normal decode])
>         Capabilities: [140 v1] Access Control Services
> 00:1c.1 0604: 8086:a111 (rev f1) (prog-if 00 [Normal decode])
>         Capabilities: [140 v1] Access Control Services
> 00:1c.2 0604: 8086:a112 (rev f1) (prog-if 00 [Normal decode])
>         Capabilities: [140 v1] Access Control Services
> 00:1c.3 0604: 8086:a113 (rev f1) (prog-if 00 [Normal decode])
>         Capabilities: [140 v1] Access Control Services
> 00:1c.6 0604: 8086:a116 (rev f1) (prog-if 00 [Normal decode])
>         Capabilities: [140 v1] Access Control Services
>
> I have tried [proxmox ve 4.0, proxmox 4.1, centos 7, ubuntu 1404, ubuntu
> 1510] + add "pci_acs_override=downstream" to kernel parameters.
> But the iommu group 4 is always the same.
>

The ACS override patch is not upstream, you'd actually need to run a kernel
with support for it for the kernel command line option to work.  You have a
Sunrise Point PCH where Intel has a broken implementation of ACS.  I've
posted quirks[1] for this, but they probably won't be available until a
v4.7 kernel.  Your choices are to either patch your kernel with the ACS
override patch or patch your kernel with the quirks (preferred) until
support is merged into the kernel.  Thanks,

Alex

[1] https://lkml.org/lkml/2016/3/31/1112
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160405/a1dbbd3d/attachment.htm>