[Libguestfs] virt-builder fails in fedora 34 container:

[Nir Soffer]

On Sun, Jun 13, 2021 at 1:38 PM Richard W.M. Jones <rjones at redhat.com> wrote:
>
> On Sat, Jun 12, 2021 at 11:11:22PM +0300, Nir Soffer wrote:
> > I'm running this inside a fedora 34 container in ovirt-imageio CI:
> >
> >     'virt-builder', '-v', '-x', 'cirros-0.3.5', '--write',
> > '/etc/cirros-init/config:DATASOURCE_LIST="nocloud"',
> > '--root-password', 'password:', '-o',
> > '/var/tmp/imageio-images/cirros-0.3.5'
> >
> > The command fails in:
> >
> > supermin: ext2: copying kernel modules
> > supermin: ext2: creating minimal initrd
> > '/var/tmp/.guestfs-0/appliance.d.t7nny0l7/initrd'
> > supermin: exception:
> > Sys_error("/lib/modules/5.12.9-300.fc34.x86_64/modules.dep: No such
> > file or directory")
>
> We use this file in order to construct the minimal initrd with some
> virtio modules, which is sufficient to bring up the supermin root
> filesystem.  The file itself is used to calculate the correct set of
> module dependencies (so that, for example, if a virtio kmod needs
> another kmod then the other kmod will be pulled in).  See:
>
> https://github.com/libguestfs/supermin/blob/c89e4923909ef84541ac1c047bf64d13b1595b4f/src/format_ext2_initrd.ml
>
> I believe in Fedora the file is created via a %post script in
> kernel-core which runs /bin/kernel-install (from systemd-udev).  This
> in turn is a shell script which runs snippets from
> /etc/kernel/install.d and /usr/lib/kernel/install.d which are supplied
> from all kinds of other packages.  On my machine one of these is
> /usr/lib/kernel/install.d/50-depmod.install (also from systemd-udev)
> which eventually runs depmod -a which creates this file.
>
> So those are some of the packages which might be missing.  Does
> modules.dep get created anywhere inside the container?  It's possible
> it moved.
>
> It's also possible to set the environment variable KERNEL_INSTALL_VERBOSE=1
> which should show (in the dnf install output I guess?) if/when the
> depmod command runs.
>
> > I think I'm missing some package removed from the fedora 34 container,
> > but I don't have
> > any idea which one.
> >
> > I hope someone has more insight on this.
>
> It's curious that things have changed since F33.  Could you build
> containers for both and diff the list of package names?  Maybe a
> dependency is missing or has become a soft dep.
>
> rpm -qa --qf '%{name}\n' | sort

I ran this:

$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-34 rpm -qa
--qf '%{name}\n' | sort > fedora-34.out
$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-33 rpm -qa
--qf '%{name}\n' | sort > fedora-33.out
$ diff -u fedora-33.out fedora-34.out > fedora.diff
$ egrep -- ^- fedora.diff > removed-in-fedora-34
$ egrep -- '^[+]' fedora.diff > added-in-fedora-34

$ cat removed-in-fedora-34
--- fedora-33.out 2021-06-13 22:51:13.156150245 +0300
-autogen-libopts
-genisoimage
-iptables
-isl
-kbd-legacy
-libcbor
-libdb-utils
-libfido2
-libguestfs-tools-c
-libmodman
-libusal
-nmap-ncat
-pciutils
-pciutils-libs
-python3-idna
-rdma-core
-selinux-policy-minimum

$ cat added-in-fedora-34
+++ fedora-34.out 2021-06-13 22:50:59.650016271 +0300
+guestfs-tools
+inih
+iptables-legacy
+iptables-legacy-libs
+langpacks-core-en_GB
+langpacks-en_GB
+libbsd
+libcanberra
+libcanberra-gtk3
+libcloudproviders
+libpmemobj
+libretls
+libtdb
+netcat
+pcsc-lite-libs
+selinux-policy-targeted
+sound-theme-freedesktop

I don't see anything that can explain the difference.

I also compared contents of the /lib/modules/xxx/ directories:

$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-33 ls -1
/lib/modules/5.12.9-200.fc33.x86_64 >
fedora-33-lib_modules_5.12.9-200.fc33.x86_64.out
$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-34 ls -1
/lib/modules/5.12.9-300.fc34.x86_64 >
fedora-34-lib_modules_5.12.9-300.fc34.x86_64.out
$ diff -u fedora-33-lib_modules_5.12.9-200.fc33.x86_64.out
fedora-34-lib_modules_5.12.9-300.fc34.x86_64.out
--- fedora-33-lib_modules_5.12.9-200.fc33.x86_64.out 2021-06-13
23:12:04.177539132 +0300
+++ fedora-34-lib_modules_5.12.9-300.fc34.x86_64.out 2021-06-13
23:12:18.072675988 +0300
@@ -3,23 +3,14 @@
 build
 config
 kernel
-modules.alias
-modules.alias.bin
 modules.block
 modules.builtin
 modules.builtin.alias.bin
-modules.builtin.bin
 modules.builtin.modinfo
-modules.dep
-modules.dep.bin
-modules.devname
 modules.drm
 modules.modesetting
 modules.networking
 modules.order
-modules.softdep
-modules.symbols
-modules.symbols.bin
 source
 symvers.gz
 updates

Looks like some post install step is missing on the fedora 34 container.