[libvirt] [PATCH v2 0/4] New mdev type handling for aggregated resources

[Erik Skultety]

On Thu, Jul 26, 2018 at 08:29:45AM -0600, Alex Williamson wrote:
> On Thu, 26 Jul 2018 15:50:28 +0200
> Erik Skultety <eskultet at redhat.com> wrote:
>
> > On Tue, Jul 24, 2018 at 11:44:40AM -0600, Alex Williamson wrote:
> > > On Fri, 20 Jul 2018 10:19:24 +0800
> > > Zhenyu Wang <zhenyuw at linux.intel.com> wrote:
> > >
> > > > Current mdev device create interface depends on fixed mdev type, which get uuid
> > > > from user to create instance of mdev device. If user wants to use customized
> > > > number of resource for mdev device, then only can create new mdev type for that
> > > > which may not be flexible. This requirement comes not only from to be able to
> > > > allocate flexible resources for KVMGT, but also from Intel scalable IO
> > > > virtualization which would use vfio/mdev to be able to allocate arbitrary
> > > > resources on mdev instance. More info on [1] [2] [3].
> > > >
> > > > To allow to create user defined resources for mdev, it trys to extend mdev
> > > > create interface by adding new "instances=xxx" parameter following uuid, for
> > > > target mdev type if aggregation is supported, it can create new mdev device
> > > > which contains resources combined by number of instances, e.g
> > > >
> > > >     echo "<uuid>,instances=10" > create
> > > >
> > > > VM manager e.g libvirt can check mdev type with "aggregation" attribute which
> > > > can support this setting. If no "aggregation" attribute found for mdev type,
> > > > previous behavior is still kept for one instance allocation. And new sysfs
> > > > attribute "instances" is created for each mdev device to show allocated number.
> > > >
> > > > This trys to create new KVMGT type with minimal vGPU resources which can be
> > > > combined with "instances=x" setting to allocate for user wanted resources.
> > >
> > > "instances" makes me think this is arg helps to create multiple mdev
> > > instances rather than consuming multiple instances for a single mdev.
> > > You're already exposing the "aggregation" attribute, so doesn't
> > > "aggregate" perhaps make more sense as the create option?  We're asking
> > > the driver to aggregate $NUM instances into a single mdev.  The mdev
> > > attribute should then perhaps also be "aggregated_instances".
> > >
> > > The next user question for the interface might be what aspect of the
> > > device gets multiplied by this aggregation?  In i915 I see you're
> > > multiplying the memory sizes by the instance, but clearly the
> > > resolution doesn't change.  I assume this is sort of like mdev types
> > > themselves, ie. some degree of what a type means is buried in the
> > > implementation and some degree of what some number of those types
> > > aggregated together means is impossible to describe generically.
> >
> > I don't seem to clearly see the benefit here, so I have to ask, how is this
> > better and/or different from allowing a heterogeneous setup if one needs a more
> > performant instance in terms of more resources? Because to me, once you're able
> > to aggregate instances, I would assume that a simple "echo `uuid`" with a
> > different type should succeed as well and provide me (from user's perspective)
> > with the same results. Could you please clarify this to me, as well as what
> > resources/parameters are going to be impacted by aggregation?
>
> I think you're suggesting that we could simply define new mdev types to
> account for these higher aggregate instances, for example we can
> define discrete types that are 2x, 3x, 4x, etc. the resource count of a
> single instance.  What I think we're trying to address with this
> proposal is what happens when the resources available are exceptionally
> large and they can be combined in arbitrary ways.  For example if a
> parent device can expose 10,000 resources and the granularity with
> which we can create and mdev instance is 1, is it practical to create
> 10,000 mdev types or does it make more sense to expose a granularity
> and method of aggregation.

Okay, I got a much better picture now, thanks for the clarification.
The granularity you mentioned definitely does give users more power and control
(in terms of provisioning) over the devices.

Erik

>
> Using graphics here perhaps falls a little short of the intention of
> the interface because the possible types are easily enumerable and it
> would be entirely practical to create discrete types for each.  vGPUs
> also have a lot of variables, so defining which attribute of the device
> is multiplied by the number of instances is a little more fuzzy.
> Thanks,
>
> Alex