[libvirt] [Qemu-devel] Modern CPU models cannot be used with libvirt

[Gleb Natapov]

On Thu, Mar 22, 2012 at 12:50:18PM -0300, Eduardo Habkost wrote:
> On Thu, Mar 22, 2012 at 04:30:55PM +0200, Gleb Natapov wrote:
> > On Thu, Mar 22, 2012 at 10:31:21AM -0300, Eduardo Habkost wrote:
> > > On Thu, Mar 22, 2012 at 11:32:44AM +0200, Gleb Natapov wrote:
> > > > What does this mean? Will -nodefconfig disable loading of bios.bin,
> > > > option roms, keymaps?
> > > 
> > > Correcting myself: loading of _config_ files on /usr/share. ROM images
> > > are opaque data to be presented to the guest somehow, just like a disk
> > > image or kernel binary. But maybe keymaps will become "configuration"
> > > someday, I really don't know.
> > > 
> > Where do you draw the line between "opaque data" and configuration. CPU
> > models are also something that is present to a guest somehow.
> 
> Just the fact that it's in a structured key=value format that Qemu
> itself will interpret before exposing something to the guest. Yes, it's
> a bit arbitrary. If we could make everything "configuration data", we
> would (or that's what I think Anthony is pushing for--I hope he will
> reply and clarify that).
> 
It is not a "bit arbitrary" it is completely arbitrary.

> > Are you
> > consider ROMs to be "opaque data" because they are binary and CPU models
> > to be config just because it is ascii file?
> 
> Not just "ascii file", but structured (and relatively small)
> [section]key=value data. If BIOSes were not opaque binary code and could
> be converted to a small set of config sections and key=values just like
> cpudefs, one could argue that BIOSes could become configuration data,
> too.
> 
There is no argument I can make about it since there is no logic to
refute to begin with :) Well may be except that when cpu model file will
support configuring all couid leafs for each cpu model it will not
be so small :)
 
> 
> > What if we pre-process CPU
> > models into binary for QEMU to read will it magically stop being
> > configuration?
> 
> Doing the reverse (transforming simple [section]key=value data to a
> binary format) would just be silly and wouldn't gain us anything. The
> point here is that we (Qemu) seem to be moving towards a design where
> most things are structured "configuration data" that fits on a
> [section]key=value model, and Qemu just interprets that structured data
> to build a virtual machine.
Nothing silly about it. You can move data parsing outside of QEMU and
just mmap cpu definitions in QEMU.

> 
> (That's why I said that perhaps keymaps could become configuration
> someday. Because maybe they can be converted to a key=value model
> relatively easily)
> 
Such whole sale approach is harmful since it starts to affect design
decisions. So now if it seams logical to move something outside the code
one can decide against it just because it will become "configuration"
due to that design.

> 
> > > > > > Doing this would make it impossible to deploy fixes to users if we evern
> > > > > > find out that the default configuration file had a serious bug. What if
> > > > > > a bug in our default configuration file has a serious security
> > > > > > implication?
> > > > > 
> > > > > The answer to this is: if the broken templates/defaults are on
> > > > > /usr/share, it would be easy to deploy the fix.
> > > > > 
> > > > > So, the compromise solution is:
> > > > > 
> > > > > - We can move some configuration data (especially defaults/templates)
> > > > >   to /usr/share (machine-types and CPU models could go there). This
> > > > >   way we can easily deploy fixes to the defaults, if necessary.
> > > > > - To reuse Qemu models, or machine-types, and not define everything from
> > > > >   scratch, libvirt will have to use something like:
> > > > >   "-nodefconfig -readconfig /usr/share/qemu/cpu-models-x86.conf"
> > > > > 
> > > > cpu-models-x86.conf is not a configuration file. It is hardware
> > > > description file. QEMU should not lose capability just because you run
> > > > it with -nodefconfig. -nodefconfig means that QEMU does not create
> > > > machine for you, but all parts needed to create a machine that would have
> > > > been created without -nodefconfig are still present. Not been able to
> > > > create Nehalem CPU after specifying -nodefconfig is the same as not been
> > > > able to create virtio-net i.e the bug.
> > > 
> > > 
> > > The current design direction Qemu seems to be following is different
> > > from that: hardware description is also considered "configuration" just
> > > like actual machine configuration. Anthony, please correct me if I am
> > > wrong.
> > That's a bug. Why trying to rationalize it now instead of fixing it.
> 
> It's just a bug for you because you disagree with the current design.
> You can call it rationalization, yes; I am just accepting Anthony's
> proposal because it's equally good (to me) as what you are proposing.
> 
> 
> > It
> > was fixed in RHEL by the same person who introduced it in upstream in
> > the first place. He just forgot to send the fix upstream. Does bug that
> > is present for a long time is promoted to a feature?
> 
> John didn't forget it, he knew that upstream could go in a different
> direction. The RHEL6 patch description has this:
> 
> "Note this is intended as an interim work-around for rhel6.0. While the
> new location of the config file should remain the same, the mechanism to
> direct qemu to it will likely differ going forward."
> 
That's even worse. It is very sad state of affairs for upstream if he
knew that the fix will not be accepted. As a result QEMU upstream does
not work properly with libvirt for actually users.

> > > > > (the item below is not something discussed on the call, just something I
> > > > > want to add)
> > > > > 
> > > > > To make this work better, we can allow users (humans or machines) to
> > > > > "extend" CPU models on the config file, instead of having to define
> > > > > everything from scratch. So, on /etc (or on a libvirt-generated config)
> > > > > we could have something like:
> > > > > 
> > > > > =============
> > > > > [cpu]
> > > > > base_cpudef = Nehalem
> > > > > add_features = "vmx"
> > > > > =============
> > > > > 
> > > > > Then, as long as /usr/share/cpu-models-x86.conf is loaded, the user will
> > > > > be able to reuse the Nehalem CPU model provided by Qemu.
> > > > > 
> > > > And if it will not be loaded?
> > > 
> > > If it is not loaded, it is a configuration mistake. If you are reusing
> > > something defined somewhere, it would be your responsibility to make
> > > sure the file where the model is defined is present. On most cases you
> > > wouldn't use -nodefconfig and it would be shipped with Qemu and you
> > > shouldn't worry. If you used -nodefconfig, you load the CPU models file
> > > explicitly using -readconfig.
> > > 
> > Regular user has no idea that we describe some cpu models in .c code and
> > others in config file and those that are described in config files are
> > disappear if  -nodefconfig is used and that if they disappear they
> > should be loaded back and how exactly they should be loaded back. The
> > only user of -nodefconfig is libvirt and currently it does unexpected
> > things for its only user.
> 
> And one could argue that this only user has wrong expectations and Qemu
> does it this way by design, and we would be running in circles forever.
QEMU is new GNOME3?

> The only problem for us is that if we keep running in circles, Qemu will
> keep the current design (or bug, if you want to call it), until Qemu
> maintainers agree to change the current behavior.
> 
> I mean: you're right into arguing for it, but somehow I feel I am the
> wrong person to reply to your arguments, because IMO both approaches are
> valid. I really hope Anthony will reply to your points, too, or that we
> discuss that on the next Qemu call. Because if you convince only me, we
> would gain nothing if the patches implementing the design we agreed with
> get rejected. It's really a pity that your arguments weren't exposed
> last week during the Qemu call, when I and Anthony discussed this and
> agreed on the "compromise solution" I described.
> 
If I will convince you we can both convince Anthony afterwords :) You
are keeping rationalizing the current behaviour, and since you are the
only one who is doing it I have no other points to reply to.

> 
> > > > > > > 
> > > > > > > But now when libvirt uses -nodefconfig, those models go away.
> > > > > > > -nodefconfig means start QEMU in the most minimal state possible.
> > > > > > > You get what you pay for if you use it.
> > > > > > > 
> > > > > > > We'll have the same problem with machine configuration files.  At
> > > > > > > some point in time, -nodefconfig will make machine models disappear.
> > > > > > 
> > > > > > It shouldn't. Machine-types are defaults to be used as base, they are
> > > > > > not user-provided configuration. And the fact that we decided to store
> > > > > > some data outside of the Qemu binary is orthogonal the design decisions
> > > > > > in the Qemu command-line and configuration interface.
> > > > > 
> > > > > So, this problem is solved if the defaults are easily found on
> > > > > /usr/share.
> > > > > 
> > > > What problem is solved and why are we mixing machine configuration files
> > > > and cpu configuration files? They are different and should be treated
> > > > differently.
> > > 
> > > This is the root of the disagreement, it seems: they are not considered
> > > different today. Today cpudefs are on a config file inside /etc.  One
> > > may argue that this was a mistake in the first place, but that's the
> > > design we have today.
> > > 
> > Machine configuration files do not exist today! Machine configuration is
> > done in .c code and -nodefconfig skips the code that does that, but it
> > does not skip the code that describes cpu models and resides in .c. So
> > as far as .c code is concerned machine creation and cpu description are
> > two different things, but for some reason the file with rest of cpu
> > model descriptions is not loaded. This is madness, not design.
> 
> Let's try to agree on terminology here: what's "machine configuration"?
> I understand it as the configuration of a specific virtual machine for a
> specific Qemu instance (i.e. what we put on the command-line today), I
> am not talking about machine-types (yet ;).
> 
> So, considering this definition, machine configuration exists today,
> there is stuff you can define on a configuration file, and you can give
> this configuration to -readconfig today. The config file is
My /etc/qemu/ contains no machine configuration file after fresh QEMU
install. The only thing there is the thing that shouldn't be there: cpu
modes definition. So everything I wrote above is correct. i.e -nodefconfig
skips the _code_ that creates machine (not reading a config file) and it
does not skip cpu model definitions that are done in .c file. I would
gladly hear rationalization for this behaviour and you just ignored it
in your reply.

> unfortunately not as powerful as the command-line (some things can be
> set only on the command-line today), but it exists. Anthony even
> submitted a RFC series to make the config files as powerful as the
> command-line, by adding a [system] section. I don't agree completely
> with that specific implementation, but I agree with the direction it is
> taking.
> 
> 
> > > 
> > > > -nodefconfig exists only because there is not machine
> > > > configuration files currently. With machine configuration files
> > > > libvirt does not need -nodefconfig because it can create its own machine
> > > > file and make QEMU use it. So specifying machine file on QEMU's command
> > > > line implies -nodefconfig. The option itself loses its meaning and can be
> > > > dropped.
> > > 
> > > I think the approach today is:
> > There is not such thing as todays approach since there is not machine
> > config file today.
> 
> There are config files today, and they work (but they are not as
> powerful as the command-line). But the interface is the one I describe
> below.
Again, this is not how default machine is created, so hardly relevant to
-nodefconfig behaviour, no?

> 
> > 
> > > 
> > > - Qemu loads defaults from default config files;
> > > - Machine description files would be given using -readconfig and they
> > >   would _augment_ the defaults from the default config files.
> > Jut make it possible to include one machine config file from another and
> > you do not need  -nodefconfig again.
> 
> That would be a valid approach too, but this is just not how the config
> files work today. We may change it, yes. But today the config file
> system is based on getting config data from multiple places
> (/etc/qemu/qemu.conf, extend it with /etc/qemu/target-x86_64.conf, and
> extend the result with the command-line arguments), instead of using
> includes.
Nothing wrong with that! But cpu models should not be part of
/etc/qemu/target-x86_64.conf :)

> 
> > But this is completely orthogonal to
> > the discussion if cpu models are configuration or not.
> 
> You are right. How the "machine configuration" is going to be loaded is
> orthogonal to the main question. The main question here is wheter
> cpudefs should be included on what we call "machine configuration" or
> not. (agreed?)
> 
> -- 
> Eduardo

--
			Gleb.