[Libguestfs] [PATCH virt-v2v] v2v: Allow temporary directory to be set on a global basis.

Tue Apr 7 14:31:57 UTC 2020

On Tuesday, 7 April 2020 14:59:11 CEST Tomáš Golembiovský wrote:
> On Tue, Apr 07, 2020 at 02:33:20PM +0200, Pino Toscano wrote:
> > On Tuesday, 7 April 2020 14:18:47 CEST Richard W.M. Jones wrote:
> > > On Tue, Apr 07, 2020 at 01:25:02PM +0200, Pino Toscano wrote:
> > > > The important thing is still that that you need to have space for the
> > > > temporary files somewhere: be it /var/tmp, /mnt/scratch, whatever.
> > > > Because of this, and the fact that usually containers are created
> > > > fresh, the cache of the supermin appliance starts to make little sense,
> > > > and then a very simple solution is to point libguestfs to that extra
> > > > space:
> > > > 
> > > >   $ dir=$(mktemp --tmpdir -d /path/to/big/temporary/space/libguestfs.XXXXXX)
> > > >   $ export LIBGUESTGS_CACHEDIR=$dir
> > > >   $ export TMPDIR=$dir  # optionally
> > > >   $ libguestfs-test-tool
> > > >   [etc]
> > > >   $ rm -rf $dir
> > > > 
> > > > Easy to use, already doable, solves all the issues.
> > > 
> > > So AIUI there are a few problems with this (although I'm still
> > > investigating and trying to make a local reproducer):
> > > 
> > >  - The external large space may be on NFS, with the usual problems
> > >    there like root_squash, no SELinux labels, slowness.  This means
> > >    it's not suitable for the appliance, but might nevertheless be
> > >    suitable for overlay files.
> > 
> > If that is the only big storage space attached to a container, I do
> > not see any alternative than use it, with all the caveats associated.
> 
> I have to aggree with this. You cannot avoid situations where the
> appliance is on a network drive. If there are any inherent risks the
> best you can do is let user know about those (documentation?).
> 
> > 
> > Also, if we take as possible scenario the situation where /var/tmp is
> > not that big, then we need to consider that may not be big enough to
> > even store the cached supermin appliance (which is more than
> > 300/350MB).
> > 
> > >  - The external large space may be shared with other containers, and
> > >    I'm not convinced that our locking in supermin will be safe if
> > >    multiple parallel instances start up at the same time.  We
> > >    certainly never tested it, and don't currently advise it.
> > 
> > That's why my suggestion above creates a specific temporary directory
> > for each container: even with a shared /var/tmp, there will not be any
> > cache stepping up on each other toes. This is something that this
> > separate cachedir for virt-v2v does not solve at all.
> 
> Currently we don't share the temporary volume between instances in
> Kubevirt, but the assumption that this can happen is valid and should be
> considered.
> 
> > 
> > > > This whole problem started from a QE report on leftover files after
> > > > failed migrations: bz#1820282.
> > > 
> > > (I should note also there are two bugs which I personally think we can
> > > solve with the same fix, but they are completely different bugs.)
> > 
> > I still do not understand how these changes have anything to do with
> > bug 1814611, which in an offline discussion we found out that has
> > mostly two causes:
> > - the way the NFS storage is mounted over the /var/tmp in the
> >   container, so what you create as root is not really with the UID/GID
> >   expected
> > - the fixed appliance in the container was not actually used, and thus
> >   a rebuilt of the the supermin appliance was attempted, failing due
> >   to the first issue
> 
> I am still not convinced this is the case. Based on the logs I shared in
> private email I still believe that the fixed appliance was used
> properly. You assumed that the appliance is not used because the cache
> directory is being created, but as I also pointed out the cache
> directory created in all situations because qemu temporary files are
> stored there [1][2].

This may be the case, i.e. something else before even the supermin
appliance check that triggers the creation of the cachedir.

In the end though, libguestfs prefers a supermin appliance before a
fixed appliance; the whole logic is here:
https://github.com/libguestfs/libguestfs/blob/c2c11382bbeb4500f3388a31ffd08cfc18b0de40/lib/appliance.c
In particular, see the locate_or_build_appliance function and the
comment before it, and contains_fixed_appliance &
contains_supermin_appliance helper functions.

If you have a setup like:

/usr/lib64/guestfs
├── initrd
├── kernel
├── README.fixed
├── root
└── supermin.d
    ├── base.tar.gz
    └── packages

with LIBGUESTFS_PATH=/usr/lib64/guestfs, then the logic fill find first
the two files under supermin.d as supermin appliance, and not even
consider the fixed appliance there.

> 
> > 
> > Can you please explain me exactly how switching the location of
> > temporary files (that were not mentioned in the bug at all) will fix
> > this situation?
> 
> For the particular BZ 1814611, if we keep the fixed appliance we can
> move the cache directory to something else than /var/tmp (just for the
> qemu files).

Or simply move the location of the fixed appliance, and set
LIBGUESTFS_PATH to it. I would do it regardless to avoid the situation
described above, i.e. that the fixed appliance is shadowed by the
supermin appliance because of the same location of both.

> > > > What this report doesn't say, however,
> > > > is that beside the mentioned files that virt-v2v creates, there are
> > > > also leftover files that libguestfs itself creates. These files are
> > > > usually downloaded from the guest for the inspection, and generally not
> > > > that big compared to e.g. the overlays that virt-v2v creates.
> > > > Nonetheless, an abrupt exit of virt-v2v will leave those in place, and
> > > > they will still slowly fill up the space on /var/tmp (or whatever is
> > > > the location of $LIBGUESTFS_CACHEDIR).
> > > 
> > > I guess that small files being left around aren't really a problem.
> > > The problem they have is large files being left around, and I can
> > > understand why that would be an issue and not the small files.
> > 
> > Nobody is saying that the leftover files are not a problem. I'm saying
> > that also the small files are a sort of problem -- sure, less critical,
> > however still there and ready to show up any time, especially if the
> > concern is the space of /var/tmp.
> 
> Isn't that the reason why things like tmpreaper exist?

Then we can apply that also for the virt-v2v files in general, running
it for any content older than 2/3 days. Even better, with the proposed
consolidation of temporary directories [1], tmpreaper can be pointed at
those directories only, not touching anything else.

[1] https://www.redhat.com/archives/libguestfs/2020-April/msg00050.html

-- 
Pino Toscano
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20200407/0d92e5d9/attachment.sig>