[libvirt] mdevctl: A shoestring mediated device management and persistence utility

[Sylvain Bauza]

On Tue, Jun 18, 2019 at 1:01 PM Cornelia Huck <cohuck at redhat.com> wrote:

> On Mon, 17 Jun 2019 11:05:17 -0600
> Alex Williamson <alex.williamson at redhat.com> wrote:
>
> > On Mon, 17 Jun 2019 16:10:30 +0100
> > Daniel P. Berrangé <berrange at redhat.com> wrote:
> >
> > > On Mon, Jun 17, 2019 at 08:54:38AM -0600, Alex Williamson wrote:
> > > > On Mon, 17 Jun 2019 15:00:00 +0100
> > > > Daniel P. Berrangé <berrange at redhat.com> wrote:
> > > >
> > > > > On Thu, May 23, 2019 at 05:20:01PM -0600, Alex Williamson wrote:
>
> > > > > > Hi,
> > > > > >
> > > > > > Currently mediated device management, much like SR-IOV VF
> management,
> > > > > > is largely left as an exercise for the user.  This is an attempt
> to
> > > > > > provide something and see where it goes.  I doubt we'll solve
> > > > > > everyone's needs on the first pass, but maybe we'll solve enough
> and
> > > > > > provide helpers for the rest.  Without further ado, I'll point
> to what
> > > > > > I have so far:
> > > > > >
> > > > > > https://github.com/awilliam/mdevctl
> > > > > >
> > > > > > This is inspired by driverctl, which is also a bash utility.
> mdevctl
> > > > > > uses udev and systemd to record and recreate mdev devices for
> > > > > > persistence and provides a command line utility for querying,
> listing,
> > > > > > starting, stopping, adding, and removing mdev devices.
> Currently, for
> > > > > > better or worse, it considers anything created to be
> persistent.  I can
> > > > > > imagine a global configuration option that might disable this and
> > > > > > perhaps an autostart flag per mdev device, such that mdevctl
> might
> > > > > > simply "know" about some mdevs but not attempt to create them
> > > > > > automatically.  Clearly command line usage help, man pages, and
> > > > > > packaging are lacking as well, release early, release often,
> plus this
> > > > > > is a discussion starter to see if perhaps this is sufficient to
> meet
> > > > > > some needs.
> > > > >
> > > > > I think from libvirt's POV, we would *not* want devices to be made
> > > > > unconditionally persistent. We usually wish to expose a choice to
> > > > > applications whether to have resources be transient or persistent.
> > > > >
> > > > > So from that POV, a global config option to turn off persistence
> > > > > is not workable either. We would want control per-device, with
> > > > > autostart control per device too.
> > > >
> > > > The code has progressed somewhat in the past 3+ weeks, we still
> persist
> > > > all devices, but the start-up mode can be selected per device or
> with a
> > > > global default mode.  Devices configured with 'auto' start-up
> > > > automatically while 'manual' devices are simply known and available
> to
> > > > be started.  I imagine we could add a 'transient' mode where we purge
> > > > the information about the device when it is removed or the next time
> > > > the parent device is added.
> > >
> > > Having a pesistent config written out & then purged later is still
> > > problematic. If the host crashes, nothing will purge the config file,
> > > so it will become a persistent device. Also when listing devices we
> > > want to be able to report whether it is persistent or transient. The
> > > obvious way todo that is to simply look if a config file exists or
> > > not.
> >
> > I was thinking that the config file would identify the device as
> > transient, therefore if the system crashed we'd have the opportunity to
> > purge those entries on the next boot as we're processing the entries
> > for that parent device.  Clearly it has yet to be implemented, but I
> > expect there are some advantages to tracking devices via a transient
> > config entry or else we're constantly re-discovering foreign mdevs.
>
> I think we need to reach consensus about the actual scope of the
> mdevctl tool.
>
>
Thanks Cornelia, my thoughts:

- Is it supposed to be responsible for managing *all* mdev devices in
>   the system, or is it more supposed to be a convenience helper for
>   users/software wanting to manage mdevs?
>

The latter. If an operator (or some software) wants to create mdevs by not
using mdevctl (and rather directly calling the sysfs), I think it's OK.
That said, mdevs created by mdevctl would be supported by systemctl, while
the others not but I think it's okay.

- Do we want mdevctl to manage config files for individual mdevs, or
>   are they supposed to be in a common format that can also be managed
>   by e.g. libvirt?
>

Unless I misunderstand, I think mdevctl just helps to create mdevs for
being used by guests created either by libvirt or QEMU or even others.
How a guest would allocate a mdev (ie. saying "I'll use this specific mdev
UUID") is IMHO not something for mdevctl.

- Should mdevctl be a stand-alone tool, provide library functions, or
>   both? Related: should it keep any internal state that is not written
>   to disk? (I think that also plays into the transient vs. persistent
>   question.)
>
>
FWIW, I'd love using mdevctl for OpenStack (Nova) just at least for
creating persisted mdevs (ie. mdevs that would be recreated after rebooting
using systemctl). That's the real use case I need.
Whether libvirt would internally support mdevctl would be nice but that's
not really something Nova needs, so I leave others providing their own
thoughts.


My personal opinion is that mdevctl should be able to tolerate mdevs
> being configured by other means, but probably should not try to impose
> its own configuration if it detects that (unless explicitly asked to do
> so). Not sure how feasible that goal is.
>
> That's what I misunderstand : in order to have a guest using a vGPU, you
need to do two things :
1/ create the mdev
2/ allocate this created dev to a specific guest config

Of course, we could imagine a way to have both steps to be done directly by
libvirt, but from my opinion, mdevctl is really helping 1/ and not 2/.
-Sylvain



> A well-defined config file format is probably a win, even if it only
> ends up being used by mdevctl itself.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20190618/e63d9822/attachment-0001.htm>