[libvirt] [PATCH v2 0/4] New mdev type handling for aggregated resources

[Alex Williamson]

On Thu, 26 Jul 2018 15:50:28 +0200
Erik Skultety <eskultet at redhat.com> wrote:

> On Tue, Jul 24, 2018 at 11:44:40AM -0600, Alex Williamson wrote:
> > On Fri, 20 Jul 2018 10:19:24 +0800
> > Zhenyu Wang <zhenyuw at linux.intel.com> wrote:
> >  
> > > Current mdev device create interface depends on fixed mdev type, which get uuid
> > > from user to create instance of mdev device. If user wants to use customized
> > > number of resource for mdev device, then only can create new mdev type for that
> > > which may not be flexible. This requirement comes not only from to be able to
> > > allocate flexible resources for KVMGT, but also from Intel scalable IO
> > > virtualization which would use vfio/mdev to be able to allocate arbitrary
> > > resources on mdev instance. More info on [1] [2] [3].
> > >
> > > To allow to create user defined resources for mdev, it trys to extend mdev
> > > create interface by adding new "instances=xxx" parameter following uuid, for
> > > target mdev type if aggregation is supported, it can create new mdev device
> > > which contains resources combined by number of instances, e.g
> > >
> > >     echo "<uuid>,instances=10" > create
> > >
> > > VM manager e.g libvirt can check mdev type with "aggregation" attribute which
> > > can support this setting. If no "aggregation" attribute found for mdev type,
> > > previous behavior is still kept for one instance allocation. And new sysfs
> > > attribute "instances" is created for each mdev device to show allocated number.
> > >
> > > This trys to create new KVMGT type with minimal vGPU resources which can be
> > > combined with "instances=x" setting to allocate for user wanted resources.  
> >
> > "instances" makes me think this is arg helps to create multiple mdev
> > instances rather than consuming multiple instances for a single mdev.
> > You're already exposing the "aggregation" attribute, so doesn't
> > "aggregate" perhaps make more sense as the create option?  We're asking
> > the driver to aggregate $NUM instances into a single mdev.  The mdev
> > attribute should then perhaps also be "aggregated_instances".
> >
> > The next user question for the interface might be what aspect of the
> > device gets multiplied by this aggregation?  In i915 I see you're
> > multiplying the memory sizes by the instance, but clearly the
> > resolution doesn't change.  I assume this is sort of like mdev types
> > themselves, ie. some degree of what a type means is buried in the
> > implementation and some degree of what some number of those types
> > aggregated together means is impossible to describe generically.  
> 
> I don't seem to clearly see the benefit here, so I have to ask, how is this
> better and/or different from allowing a heterogeneous setup if one needs a more
> performant instance in terms of more resources? Because to me, once you're able
> to aggregate instances, I would assume that a simple "echo `uuid`" with a
> different type should succeed as well and provide me (from user's perspective)
> with the same results. Could you please clarify this to me, as well as what
> resources/parameters are going to be impacted by aggregation?

I think you're suggesting that we could simply define new mdev types to
account for these higher aggregate instances, for example we can
define discrete types that are 2x, 3x, 4x, etc. the resource count of a
single instance.  What I think we're trying to address with this
proposal is what happens when the resources available are exceptionally
large and they can be combined in arbitrary ways.  For example if a
parent device can expose 10,000 resources and the granularity with
which we can create and mdev instance is 1, is it practical to create
10,000 mdev types or does it make more sense to expose a granularity
and method of aggregation.

Using graphics here perhaps falls a little short of the intention of
the interface because the possible types are easily enumerable and it
would be entirely practical to create discrete types for each.  vGPUs
also have a lot of variables, so defining which attribute of the device
is multiplied by the number of instances is a little more fuzzy.
Thanks,

Alex