[Libguestfs] Splitting the large libguestfs repo

[Richard W.M. Jones]

On Wed, Oct 16, 2019 at 01:57:44PM +0200, Pino Toscano wrote:
> TBH I would have concentrated on virt-v2v only, just like I handled
> virt-p2v on its own.

I've only done virt-v2v.  I've actually now got it working, including
the make check and make check-slow (although there are absolutely
bound to be latent bugs).  The new repo is here:

  https://github.com/libguestfs/virt-v2v

I'm going to add all the contributors from libguestfs.git so they have
permissions on this repo in a minute.  I've not removed the v2v/
subdirectory from the old repo.

> > You're right that people don't care about what a tool is written in,
> > so another idea might be to put the C _and_ OCaml tools into the
> > guestfs-tools.git repo (leaving lib + daemon + language bindings only
> > in libguestfs.git).  In fact I like this better now I think about it.
> 
> Still some of the questions I wrote above apply: what do we gain from
> splitting the tools in an own repository? Having them in an own
> repository means that, unles you use the API yourself (in C/Python/etc)
> then you need to build another repository to do image
> manipulation/customization.
> 
> The tools (other than v2v) are quite stable, and they do not have many
> changes these days. Looking at the release notes for libguestfs 1.38,
> and 1.40 (the latest two stable series), the biggest changes were
> - add --key in all the tools
> - output selection for --machine-readable in OCaml tools
> whose main implementations were in common code anyway.

Well we can park this for a while.  I'd still value a smaller
libguestfs.git repo in the long term.  However getting virt-v2v out
was the main priority because as you say it moves much faster than the
tools and we want to make bigger changes.

> This also crosses another topic: how is virt-v2v going to be versioned,
> and released? Similar question for the potential split of the tools.

So at the moment, virt-v2v has the same version (1.41.6).  It should
build a separate virt-v2v-1.41.X.tar.gz tarball (not actually tested
this yet).  It requires libguestfs >= 1.41.5, although in fact it
would work with libguestfs 1.40, except for literally a single test:

  https://github.com/libguestfs/virt-v2v/blob/5f355c2952729581f1b988297fe8cabe756c01e2/v2v/test-v2v-o-json.sh#L54

(I might modify that test so it is conditional on the version of
libguestfs found, and then make virt-v2v depend on libguestfs >= 1.40)

Future versioning -- up in the air.  What did we decide about virt-p2v?

> If the answer is "packed together in a single tarball like now", then
> splitting makes the work more complex only to developers, i.e. us.

I wasn't sure if you meant the common/ subdirectory here, but I'll
answer about the common/ subdirectory: It is packed into both the
libguestfs and virt-v2v tarballs.  The submodule concept doesn't exist
outside git.

> > The alternatives to the submodule are somehow packaging everything up
> > into a library, which would then become a build dep of the tools.
> > This isn't particularly nice because (as you say) it's a big mix of
> > miscellaneous "stuff", essentially a core library of missing C and
> > OCaml functionality that we happened to need.  So therefore not useful
> > as an independent library.
> 
> There is also the fact that this set of common code has no concept of
> compatibility, so this potentially needs branches for each stable
> series. Also, this crosses the topic above of releasing, w.r.t. bundled
> or independent releases.

So several things here:

Every time we significantly change libguestfs-common.git we will need
to update libguestfs.git and virt-v2v.git.  This isn't great.

Common isn't versioned, and I don't believe it needs to be because of
the previous point.

common/ subdirectory is bundled in the tarballs, so this shouldn't
affect anyone building tarballs and downstream packagers, except that
there will be a new virt-v2v package.

> > I'm not overjoyed by git submodules, but in this particular instance I
> > think it can make sense, and there's only one of them (at the moment,
> > we could change that).
> 
> Right, I'm not thrilled by git submodules either, and so far I have a
> bad experience with them.

Oh yes, me too for sure.  I think when they can work is when you have
full control over all the git modules.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW