[Libguestfs] Cargo edition problem with Debian 11

Fri Jan 13 17:52:33 UTC 2023

There actually isn't any real problem to fix; it's purely a CI problem.

Our Cargo.toml file (and transitively, our dependencies' Cargo.toml
files) specify that we need predicates-tree and predicates-core.
Unless instructed otherwise, Cargo will attempt to fetch the latest
compatible version.  However, the latest compatible versions don't
build with the older toolchain.  That why my PR adds a Cargo.lock file
which locks predicates-tree and predicates-core to exact versions that
are known to be compatible with Rust 1.48.0.  That fixes the build in
CI.

But the thing is, a Rust library crate never ships a Cargo.lock file.
Even if it did, Cargo would ignore it when building an application
that depends on that library.  So if any of nbdkit's consumers want to
build with Rust 1.48.0, they'll have to do the same Cargo.lock dance
that I did.  We don't need to make any changes to facilitate that.
Furthermore, nbdkit's consumers probably _won't_ need to do this at
all, because predicates-tree and predicates-core are merely
dev-dependencies of nbdkit.  Our consumers won't necessarily need to
fetch them at all.

Does that make sense?  And knowing this, would you prefer to skip the
Rust build in those environments, or do the Cargo.lock thing?  I'd be
ok with either.

On Fri, Jan 13, 2023 at 10:43 AM Richard W.M. Jones <rjones at redhat.com> wrote:
>
> On Fri, Jan 13, 2023 at 09:17:29AM -0700, alan somers wrote:
> > Oh, I misread your initial mail.  I saw "Debian" in the subject line
> > and assumed you were talking about packaging.  But you weren't.  This
> > is happening in CI for every platform.
> >
> > There's a straightforward way to fix the problem for Cirrus CI, the
> > one that runs after we mirror to Github.  But it requires an extra
> > step, and I'm not sure how to configure that for the .gitlab CI stuff.
> > I'll submit a partial PR and ask for help with the gitlab CI part.
> >
> > You aren't using gitlab CI to build release artifacts, are you?
>
> We're using a rather complicated scheme called libvirt-ci which
> I don't fully understand.  (Adding Eric to CC)
>
> Re your merge request here:
>
> https://github.com/libguestfs/nbdkit/pull/19
>
> I believe we could do this more easily for debian-11 simply by adding
>
>    RUST: skip
>
> to the relevant section of ci/manifest.yml.
>
> However my main thing is could we actually fix it instead of covering
> up the problem in CI?
>
> Rich.
>
> > On Fri, Jan 13, 2023 at 8:57 AM Richard W.M. Jones <rjones at redhat.com> wrote:
> > >
> > > On Fri, Jan 13, 2023 at 08:35:08AM -0700, alan somers wrote:
> > > > The nbdkit crate.  Why does Debian need a package for it?
> > >
> > > I'm not sure -- do they have one?  I can't seem to find it.
> > >
> > > Rich.
> > >
> > > > On Fri, Jan 13, 2023 at 8:30 AM Richard W.M. Jones <rjones at redhat.com> wrote:
> > > > >
> > > > > On Fri, Jan 13, 2023 at 08:25:28AM -0700, alan somers wrote:
> > > > > > IMHO this is a design bug in Debian.  Sure, they need to create
> > > > > > packages for any Rust crate that needs to install executable or .so
> > > > > > files.  But crates like nbdkit don't install anything.  They're only
> > > > > > used to build other Rust crates, and only used at compile time.  They
> > > > > > can simply be fetched during the package building process.  I really
> > > > > > don't know what Debian is doing by packaging this crate.  Is it
> > > > > > actually trying to install the ramdisk example?
> > > > >
> > > > > I'm not sure I understand - which crate?
> > > > >
> > > > > Rich.
> > > > >
> > > > > > On Fri, Jan 13, 2023 at 7:27 AM Richard W.M. Jones <rjones at redhat.com> wrote:
> > > > > > >
> > > > > > > On Fri, Jan 13, 2023 at 07:17:14AM -0700, alan somers wrote:
> > > > > > > > I think we should downgrade predicates-tree to 1.0.5 or older.  I can
> > > > > > > > submit a PR for that.  What does "rustc --version" show you?
> > > > > > >
> > > > > > > Unfortunately our configure script doesn't collect that information
> > > > > > > (it should do! - I'll fix that in a moment).
> > > > > > >
> > > > > > > But based on it being Debian 11 ("bullseye") it's likely to be:
> > > > > > >
> > > > > > > https://packages.debian.org/bullseye/rustc
> > > > > > > Package: rustc (1.48.0+dfsg1-2)
> > > > > > >
> > > > > > > Which is pretty ancient ...
> > > > > > >
> > > > > > > Rich.
> > > > > > >
> > > > > > > > On Fri, Jan 13, 2023 at 2:41 AM Richard W.M. Jones <rjones at redhat.com> wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > https://gitlab.com/nbdkit/nbdkit/-/jobs/3598390121
> > > > > > > > >
> > > > > > > > > ----------------------------------------------------------------------
> > > > > > > > > cargo build --release --example ramdisk
> > > > > > > > >  Downloading crates ...
> > > > > > > > >   Downloaded float-cmp v0.9.0
> > > > > > > > >   Downloaded downcast v0.11.0
> > > > > > > > >   Downloaded mockall_derive v0.11.3
> > > > > > > > >   Downloaded itertools v0.10.5
> > > > > > > > >   Downloaded aho-corasick v0.7.20
> > > > > > > > >   Downloaded predicates-tree v1.0.7
> > > > > > > > > error: failed to parse manifest at `/root/.cargo/registry/src/github.com-1ecc6299db9ec823/predicates-tree-1.0.7/Cargo.toml`
> > > > > > > > > Caused by:
> > > > > > > > >   failed to parse the `edition` key
> > > > > > > > > Caused by:
> > > > > > > > >   this version of Cargo is older than the `2021` edition, and only supports `2015` and `2018` editions.
> > > > > > > > > ----------------------------------------------------------------------
> > > > > > > > >
> > > > > > > > > nbdkit itself uses edition = 2018, but this seems to affects one of
> > > > > > > > > the dependencies.
> > > > > > > > >
> > > > > > > > > I'm not sure how to solve this, but one ideas I had is in ./configure
> > > > > > > > > to check if the cargo/rust we're trying to use doesn't support some
> > > > > > > > > base edition (eg. latest edition supported < 2021) then we would
> > > > > > > > > disable rust bindings.
> > > > > > > > >
> > > > > > > > > Unfortunately actually getting the latest supported edition seems
> > > > > > > > > hard.  The best I could find is parsing this which doesn't seem ideal:
> > > > > > > > >
> > > > > > > > > $ rustc --help |& grep -- --edition
> > > > > > > > >         --edition 2015|2018|2021|2024
> > > > > > > > >
> > > > > > > > > What do you think?
> > > > > > > > >
> > > > > > > > > Rich.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> > > > > > > > > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > > > > > > > > libguestfs lets you edit virtual machines.  Supports shell scripting,
> > > > > > > > > bindings from many languages.  http://libguestfs.org
> > > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> > > > > > > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > > > > > > Fedora Windows cross-compiler. Compile Windows programs, test, and
> > > > > > > build Windows installers. Over 100 libraries supported.
> > > > > > > http://fedoraproject.org/wiki/MinGW
> > > > > > >
> > > > >
> > > > > --
> > > > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> > > > > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > > > > Fedora Windows cross-compiler. Compile Windows programs, test, and
> > > > > build Windows installers. Over 100 libraries supported.
> > > > > http://fedoraproject.org/wiki/MinGW
> > > > >
> > >
> > > --
> > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> > > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > > nbdkit - Flexible, fast NBD server with plugins
> > > https://gitlab.com/nbdkit/nbdkit
> > >
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-builder quickly builds VMs from scratch
> http://libguestfs.org/virt-builder.1.html
>