[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [vfio-users] How to split iommu group on [xeon e3-1220 v5 + supermicro x11ssh-ln4f]



Dear Mr. Williamson,

Proxmox ve 4.1 has your ACS-override-patch included in the kernel.
When "pcie_acs_override=downstream" added to kernel parameter, dmesg says "PCIe ACS overrides enabled; ..."
However, iommu group 4 is still big.

I have patched the pve-kernel with your Skylake-PCH-patch.
It works now. IOMMU groups are separated. Each pcie device is in its own group.

Thank you very much.

On 4/5/2016 9:12 PM, Alex Williamson wrote:
On Tue, Apr 5, 2016 at 3:13 AM, Tran The Anh <anh tranthe thegioididong com> wrote:
Hi,

I would like to passthrough some intel i210 ports to KVM's VM, but has encountered the iommu group issue:

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:14.0 <= usb
/sys/kernel/iommu_groups/1/devices/0000:00:14.2
/sys/kernel/iommu_groups/2/devices/0000:00:16.0
/sys/kernel/iommu_groups/2/devices/0000:00:16.1
/sys/kernel/iommu_groups/3/devices/0000:00:17.0 <= sata
/sys/kernel/iommu_groups/4/devices/0000:00:1c.0 <= pch port
/sys/kernel/iommu_groups/4/devices/0000:00:1c.1 <= pch port
/sys/kernel/iommu_groups/4/devices/0000:00:1c.2 <= pch port
/sys/kernel/iommu_groups/4/devices/0000:00:1c.3 <= pch port
/sys/kernel/iommu_groups/4/devices/0000:00:1c.6 <= pch port
/sys/kernel/iommu_groups/4/devices/0000:01:00.0 <= i210
/sys/kernel/iommu_groups/4/devices/0000:02:00.0 <= i210
/sys/kernel/iommu_groups/4/devices/0000:03:00.0 <= i210
/sys/kernel/iommu_groups/4/devices/0000:04:00.0 <= i210
/sys/kernel/iommu_groups/4/devices/0000:05:00.0
/sys/kernel/iommu_groups/4/devices/0000:06:00.0 <= vga aspeed 2400
/sys/kernel/iommu_groups/5/devices/0000:00:1f.0
/sys/kernel/iommu_groups/5/devices/0000:00:1f.2
/sys/kernel/iommu_groups/5/devices/0000:00:1f.4

My system is:
Intel Xeon e3-1220 v5
Supermicro x11ssh-ln4f

According to board's diagram:
http://www.supermicro.com/manuals/motherboard/C236/MNL-1778.pdf
Those i210s are connected to PCH.

It seems that PCH ports have ACS capabilities:
lspci -vvvnk | egrep -i 'Access|Rev'
00:17.0 0106: 8086:a102 (rev 31) (prog-if 01 [AHCI 1.0])
00:1c.0 0604: 8086:a110 (rev f1) (prog-if 00 [Normal decode])
        Capabilities: [140 v1] Access Control Services
00:1c.1 0604: 8086:a111 (rev f1) (prog-if 00 [Normal decode])
        Capabilities: [140 v1] Access Control Services
00:1c.2 0604: 8086:a112 (rev f1) (prog-if 00 [Normal decode])
        Capabilities: [140 v1] Access Control Services
00:1c.3 0604: 8086:a113 (rev f1) (prog-if 00 [Normal decode])
        Capabilities: [140 v1] Access Control Services
00:1c.6 0604: 8086:a116 (rev f1) (prog-if 00 [Normal decode])
        Capabilities: [140 v1] Access Control Services

I have tried [proxmox ve 4.0, proxmox 4.1, centos 7, ubuntu 1404, ubuntu 1510] + add "pci_acs_override=downstream" to kernel parameters.
But the iommu group 4 is always the same.

The ACS override patch is not upstream, you'd actually need to run a kernel with support for it for the kernel command line option to work.  You have a Sunrise Point PCH where Intel has a broken implementation of ACS.  I've posted quirks[1] for this, but they probably won't be available until a v4.7 kernel.  Your choices are to either patch your kernel with the ACS override patch or patch your kernel with the quirks (preferred) until support is merged into the kernel.  Thanks,

Alex



Virus-free. www.avast.com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]