[Libguestfs] [PATCH 0/5] Introducing virt-builder-repository

[Cedric Bosdonnat]

On Tue, 2017-01-03 at 14:38 +0200, Nadav Goldin wrote:
> Hi, talking from user perspective here, I think this is a great addition!
> At the Lago project[1] we basically have a set of scripts that does
> something similar, but we need to maintain our own format which
> differs from the index file format in order to store few more metadata
> parameters. It would be great to have similar capability built in.

Good to read that this is a feature that would interest users ;)

> > with it: it will add extra running time to load the appliance in each
> 
> detected new/updated image
> 
> I think having this optional would be great, usually when we need to
> generate the repository we don't really care how long it takes(doesn't
> happen that frequently). What I have in mind is a similar experience
> to 'createrepo' command - just pass it a root path and the repository
> is generated. Also I think interactive mode is nice, but the more
> automated it can be the better(by pre-defining the template), maybe it
> will be possible to 'register' commands that will populate parameters?

What you mention with the automated mode is what virt-builder-repository
currently does by default. Just that is gets the missing data by parsing
a small (incomplete) index file.

As for the time it takes, I'm sure we shouldn't forget the case of
automatically built images containing updates... and this would require
virt-builder-repository to run quite often.

> My second wonder, would it be possible to add 'metadata' parameters to
> the the template used to generate the images? I know this might
> require changes to the index.asc format. Currently it has 'notes' but
> this just feels unnatural to use. For instance one use case I have in
> mind is to add a 'backing_file_hash' parameter which will indicate the
> hash of the backing_file . That way I can resolve multiple layered
> qcow images and download them just by parsing the index file.

If adding metadata is not a problem in virt-builder's index, then I'ld
happily add that support in virt-builder-repository. However I'ld rather
keep that feature away from the initial commit and have a series for the
feature implementing it in both tools at a time.

--
Cedric

> Thanks,
> 
> Nadav.
> 
> [1] https://github.com/lago-project/lago
> 
> 
> 
> On Tue, Jan 3, 2017 at 12:18 PM, Cédric Bosdonnat <cbosdonnat at suse.com> wrote:
> > Hi all,
> > 
> > I wanted to provide an easy way to create or update a virt-builder
> > repository out of a folder of template disk image files. This is what
> > virt-builder-repository aims at. Some of the data are computed from
> > the image file, others are asked the user or extracted from an existing
> > index file.
> > 
> > So far, virt-builder-repository doesn't run libguestfs on each image
> > to extract the architecture, the list of available partitions and the
> > list of volumes. May be this should be done, but I have several concerns
> > with it: it will add extra running time to load the appliance in each
> > detected new/updated image. And the question of how to handle cases where
> > multiple OSes are installed on the template would need to be solved:
> > raise an error or ask the user which one to use?
> > 
> > In order to share more code, I have extracted the Yajl helpers with the
> > yajl OCAML wrapper code, moved the libxml2 wrapper into mllib and written
> > some OCAML osinfo db reading code in mllib.
> > 
> > Cédric Bosdonnat (5):
> >   builder: extract Yajl helper functions to yajl.ml
> >   mllib: factorize code to add Checksum.get_checksum function
> >   Move xml and xpath_helpers OCAML code to mllib
> >   mllib: add libosinfo DB reading helpers
> >   Add a virt-builder-repository tool
> > 
> >  .gitignore                          |   3 +
> >  builder/Makefile.am                 |  91 ++++++-
> >  builder/builder_repository.ml       | 493 ++++++++++++++++++++++++++++++++++++
> >  builder/simplestreams_parser.ml     |  52 ----
> >  builder/virt-builder-repository.pod | 144 +++++++++++
> >  builder/yajl.ml                     |  55 ++++
> >  builder/yajl.mli                    |  29 +++
> >  docs/C_SOURCE_FILES                 |   2 +-
> >  mllib/Makefile.am                   |  19 +-
> >  mllib/checksums.ml                  |  20 +-
> >  mllib/checksums.mli                 |   3 +
> >  mllib/osinfo.ml                     |  64 +++++
> >  mllib/osinfo.mli                    |  21 ++
> >  mllib/osinfopath.ml                 |   1 +
> >  {v2v => mllib}/xml-c.c              |   0
> >  {v2v => mllib}/xml.ml               |   0
> >  {v2v => mllib}/xml.mli              |   0
> >  {v2v => mllib}/xpath_helpers.ml     |   0
> >  {v2v => mllib}/xpath_helpers.mli    |   0
> >  v2v/Makefile.am                     |  17 +-
> >  20 files changed, 932 insertions(+), 82 deletions(-)
> >  create mode 100644 builder/builder_repository.ml
> >  create mode 100644 builder/virt-builder-repository.pod
> >  create mode 100644 mllib/osinfo.ml
> >  create mode 100644 mllib/osinfo.mli
> >  create mode 100644 mllib/osinfopath.ml
> >  rename {v2v => mllib}/xml-c.c (100%)
> >  rename {v2v => mllib}/xml.ml (100%)
> >  rename {v2v => mllib}/xml.mli (100%)
> >  rename {v2v => mllib}/xpath_helpers.ml (100%)
> >  rename {v2v => mllib}/xpath_helpers.mli (100%)
> > 
> > --
> > 2.11.0
> > 
> > _______________________________________________
> > Libguestfs mailing list
> > Libguestfs at redhat.com
> > https://www.redhat.com/mailman/listinfo/libguestfs
> 
>