[libvirt] [V3] RFC for support cache tune in libvirt

[Qiao, Liyong]

Add support for cache allocation.

Thanks Martin for the previous version comments, this is the v3 version for RFC , I’v have some PoC code [2]. The follow changes are partly finished by the PoC.

#Propose Changes

## virsh command line

1. Extend output of nodeinfo, to expose L3 cache size for Level 3 (last level cache size).

This will expose how many cache on a host which can be used.

root at s2600wt:~/linux# virsh nodeinfo | grep L3
L3 cache size:       56320 KiB

2. Extend capabilities outputs.

virsh capabilities | grep resctrl
    <cpu>
    ...
      <resctrl name='L3' unit='KiB' cache_size='56320' cache_unit='2816'/>
    </cpu>

    This will tell that the host have enabled resctrl(which you can find it in /sys/fs/resctrl),
And it supports to allocate 'L3' type cache, total 'L3' cache size is 56320 KiB, and the minimum unit size of 'L3' cache is 2816 KiB.
  P.S. L3 cache size unit is the minum l3 cache unit can be allocated. It's hardware related and can not be changed.


3. Add new virsh command 'nodecachestats':
This API is to expose vary cache resouce left on each hardware (cpu socket).

It will be formated as:

<resource_type>.<resource_id>: left size KiB

for example I have a 2 socket cpus host, and I'v enabled cat_l3 feature only

root at s2600wt:~/linux# virsh nodecachestats
L3.0 : 56320 KiB
L3.1 : 56320 KiB

  P.S. resource_type can be L3, L3DATA, L3CODE, L2 for now.

4. Add new interface to manage how many cache can be allociated for a domain

root at s2600wt:~/linux# virsh cachetune kvm02 --l3.count 2

root at s2600wt:~/linux# virsh cachetune kvm02
l3.count       : 2

This will allocate 2 units(2816 * 2) l3 cache for domain kvm02

## Domain XML changes

Cache Tuneing

<domain>
  ...
  <cachetune>
    <l3_cache_count>2</l3_cache_count>
  </cachetune>
  ...
</domain>


## Restriction for using cache tune on multiple sockets' host.

The l3 cache is per socket resource, kernel need to konw about what's affinity looks like, so for a VM which running on a mulitple socket's host, it should have NUMA setting or vcpuset pin setting. Or cache tune will fail.

[1] kernel support https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/tree/arch/x86/kernel/cpu/intel_rdt.c?h=x86/cache

[2] libvirt PoC(not finished yet) https://github.com/taget/libvirt/commits/cat_new

Best Regards

Eli Qiao(乔立勇）OpenStack Core team OTC Intel.
--

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20170110/4c8c7f3f/attachment-0001.htm>