[libvirt] [RFC] Memory controller exploitation in libvirt

[Nikunj A. Dadhania]

On Tue, 24 Aug 2010 11:02:49 +0200, Matthias Bolte <matthias.bolte at googlemail.com> wrote:

<snip>

> Yes the ESX driver allows to control ballooning through
> virDomainSetMemory and virDomainSetMaxMemory.
> 
> ESX itself also allows to set what's called memoryMinGaurantee in the
> thread, but this is not exposed in libvirt.
LXC driver is using virDomainSetMemory to set the memory hard limit while
QEmu/ESX use them to change the ballooning. And as you said, ESX does support
memoryMinGaurantee, we can get this exported in libvirt using this new API.

Here I am trying to group all the memory related parameters into one single
public API as we have in virDomainSetSchedulerParameters. Currently, the names
are not conveying what they modify in the below layer and are confusing.

> So you can control how much virtual memory a guest has
> (virDomainSetMaxMemory) and define and upper (virDomainSetMemory) and
> a lower (not exposed via libvirt) bound for the physical memory that
> the hypervisor should use to satisfy the virtual memory of a guest.
> ESX also allows to defines shares, a relative value that defines a
> priority between guests in case there is not enough physical memory to
> satisfy all guests, the remaining virtual memory is then satisfied by
> swapping at the hypervisor level.
> 
> The same pattern applies to the virtual CPUs. There is an upper and a
> lower limit for the CPU allocation of a guest and a shares value to
> define priority in case of contention. All three are exposed using the
> virDomainSetSchedulerParameters API for ESX.
> 
> Regarding the new elements proposed here:
> 
>  <define name="resource">
>    <element memory>
>        <element memoryHardLimit/>
>        <element memorySoftLimit/>
>        <element memoryMinGaurantee/>
>        <element swapHardLimit/>
>        <element swapSoftLimit/>
>    </element>
>  </define>
> 
> memoryHardLimit is already there and called memory, memorySoftLimit is
> also there and called currentMemory, memoryMinGaurantee is new.
Thats correct, I am aware of this. The names are misleading. Also, we can have
all these under the memory element. 

Later we can add something like this:

<define name="resource">
  <element memory>
    <!-- All memory related tunables -->
  </element>
  <element cpu>
    <!-- All CPU related tunables -->
  </element>
  <element blkio>
    <!-- All Block IO related tunables -->
  </element>
  <element network>
    <!-- All network related tunables -->
  </element>
</define>

> I'm not sure where swapHardLimit and swapSoftLimit apply, is that for
> swapping that the hypervisor level?
Memory Cgroup provides the maximum swap a group of task can use. 
swapSoftLimit is not supported as Balbir said and they do not have plans to
support it. We can drop this.
 
> Also keep in mind that there was a recent discussion about how to
> express ballooning and memory configuration in the domain XML config:
> 
>   https://www.redhat.com/archives/libvir-list/2010-August/msg00118.html
I will have a look at this.

> Regarding future additions:
> 
>         CPUHardLimit
>         CPUSoftLimit
>         CPUShare
>         CPUPercentage
>         IO_BW_Softlimit
>         IO_BW_Hardlimit
>         IO_BW_percentage
> 
> The CPU part of this is already possible via the
> virDomainSetSchedulerParameters API. But they aren't expressed in the
> domain XML config, maybe your suggesting to do this?
Yes, thats correct for CPU. 

IO would need API as well as XML config changes. Does ESX also support Block
IO bandwidth control?

> The I/O part is in fact new, I think.
> 
> In general when you want to extend the domain XML config make sure
> that you don't model it to closely based on a specific implementation
> like CGroup.
Sure

Nikunj