[PATCH 1/4] conf: add loader type 'none'
Daniel Henrique Barboza
dbarboza at ventanamicro.com
Mon Apr 10 22:50:36 UTC 2023
On 4/7/23 15:12, Andrea Bolognani wrote:
> On Tue, Mar 28, 2023 at 04:46:45PM -0300, Daniel Henrique Barboza wrote:
>> On 3/22/23 13:37, Andrea Bolognani wrote:
>>> On Wed, Mar 22, 2023 at 10:36:20AM -0300, Daniel Henrique Barboza wrote:
>>>> I'm not sure if the OS overwrites the firmware when running bare metal. Usually
>>>> they provide different OS images for QEMU/libvirt and bare metal systems, probably
>>>> to account for these differences. Baking the firmware in the kernel like Fedora
>>>> Rawhide is doing was important a few years ago when RISC-V QEMU wasn't loading
>>>> the firmware by default, but now it's getting in the way.
>>>>
>>>> All this said, having a look at the recently updated Fedora for RISC-V wiki, the
>>>> instructions for booting with libvirt and QEMU are different. libvirt [1] is using
>>>> the '-bios none' attribute with 'virt-install' + a Fedora Rawhide image, but
>>>> QEMU instructions [2] doesn't use '-bios none' and it's using Fedora 37.
>>>
>>> The libvirt instructions in that page seem to have been updated in
>>> the Fedora 30 timeframe, whereas the QEMU instructions appear to be
>>> current as of Fedora 37.
>>>
>>>> At first I don't see any particular reason of why this Fedora 37 image would work
>>>> on QEMU and not on libvirt. So what t I'll do now is do some testings with libvirt
>>>> and Fedora 37. If it works then this series becomes way less important.
>>>
>>> The libvirt instructions tell you to use
>>>
>>> --boot kernel=Fedora-Developer-Rawhide-*-fw_payload-uboot-qemu-virt-smode.elf
>>> --qemu-commandline='-bios none'
>>>
>>> (aka -bios none -kernel foo) while the QEMU ones suggest
>>>
>>> -bios u/usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin
>>> -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
>>>
>>> Those are completely different approaches to booting the guest OS.
>>> The latter is much saner IMO, because it keeps the firmware under
>>> control of the host (as is the case for SeaBIOS/edk2) instead of
>>> mixing the kernel and the firmware and requiring guest-specific files
>>> to be extracted from the disk image before being able to boot.
>>>
>>> libvirt already provides the ability to pass arbitrary paths to -bios
>>> via the <loader type='rom'> element, so making the first part work
>>> should just be a matter of pointing it to the correct path. We don't
>>> seem to have the -device loader part wired up though, so that would
>>> need to be added. Probably as an additional attribute, in the vein of
>>> nvram.template? The address would have to be specified as well.
>>>
>>> Note that the firmware descriptor format already supports u-boot as
>>> the firmware type. So in the long run ideally you'd only need to
>>> specify
>>>
>>> <os firmware='uboot'>
>>>
>>> and, assuming the uboot-images-riscv64 package is installed on the
>>> host, everything should just work.
>>
>> So, I got the chance to try this out. We can remove the '-loader' from the
>> Fedora 37 command line. This:
>>
>> -device loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000
>>
>> Is loading the u-boot image in the address that the 'virt' machine loads the kernel.
>> So in the end it's similar to:
>>
>> -kernel u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb
>
> Excellent insight! I've confirmed that you are correct, and that
>
> <qemu:commandline>
> <qemu:arg value='-device'/>
> <qemu:arg value='loader,file=u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb,addr=0x80200000'/>
> </qemu:commandline>
>
> can indeed be replaced with
>
> <kernel>u/usr/share/uboot/qemu-riscv64_spl/u-boot.itb</kernel>
>
> without apparently affecting the VM's ability to boot.
>
>> This means that RISC-V Fedora 37 is bootable without any libvirt changes. Similar to
>> what we already do with Ubuntu.
>
> I've spent some time on this as well. Here are my findings.
>
> First, let's briefly mention how RISC-V VMs had to be booted a few
> years back. This is described in detail at [1] and preceding
> sections, but the quick version is that you would have a single file
> containing (I believe) OpenSBI, u-boot and the kernel. This file
> would need to be extracted from the disk image, and would then be
> passed to -kernel. Having the firmware and the kernel in a single
> file, which lives on the host rather than the guest, was obviously
> extremely cumbersome and unsuitable for scenarios beyond
> experimentation.
>
> Today, things are looking a lot better: the firmware lives in the
> host and the kernel lives in the guest, as they should. However,
> there seem to be two different approaches to loading the firmware
> floating around.
>
> The first one, described in the Fedora wiki[2], involves passing two
> separate files to QEMU. They both come from the uboot-images-riscv64
> package, and their use looks like
>
> -bios /usr/share/uboot/qemu-riscv64_spl/u-boot-spl.bin
> -kernel /usr/share/uboot/qemu-riscv64_spl/u-boot.itb
>
> As far as I understand, this is a u-boot binary, just divided in two
> parts and loaded separately. They might run in different hardware
> privilege contexts? I think the first file might also embed OpenSBI
> somehow.
>
> The second approach is the one described in the Ubuntu wiki[3], and
> also requires passing two files to QEMU, except this time they come
> from the opensbi and u-boot-qemu packages respectively. The usage
> looks like
>
> -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf
> -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf
>
> I think in this case the first file is a minimal build of OpenSBI
> that likely just initializes enough hardware before handing control
> to an arbitrary payload - in this case, u-boot.
>
> There's a twist, however: the Ubuntu wiki claims that the -bios part
> is not necessary with QEMU >= 7.0, and I've verified that at least
> with the QEMU 7.2 package in Fedora 37 this is accurate. I believe
> this is because opensbi-riscv64-generic-fw_dynamic.bin, which is
> included in the QEMU package, gets loaded automatically, and this
> build of OpenSBI apparently also knows how to jump ahead to where
> -kernel loads u-boot.
Yeah, until not so long ago QEMU wasn't loading any default OpenSBI firmware,
requiring the distro/user to manually add a -bios entry. Hopefully distros
can now use the default QEMU OpenSBI binary.
>
> I've tried mixing things, that is, boot the Fedora VM using the
> firmware taken from Ubuntu and vice versa, and thankfully it looks
> like this is perfectly fine.
>
> In conclusion, the current version of libvirt is perfectly capable of
> booting modern RISC-V guests through a combination of <loader> and
> <kernel> elements.
>
> That said, even if things nominally work, way too much handholding is
> necessary. So I'm wondering how we can push things forward to make
> the process more user-friendly.
>
> The obvious first step would be for the u-boot-qemu package in Ubuntu
> to ship a JSON descriptor file containing something like
>
> {
> "interface-types": [
> "u-boot"
> ],
> "mapping": {
> "device": "kernel",
> "filename": "/usr/lib/u-boot/qemu-riscv64_smode/uboot.elf"
> },
> "targets": [
> {
> "architecture": "riscv64",
> "machines": [
> "virt"
> ]
> }
> ]
> }
So, I talked with Canonical a few months back about this package. I'm using a F37
dev box and I had to uncompress the u-boot .deb package by hand to extract the
uboot.elf file to be able to launch the guest, meaning that users that aren't using
an Ubuntu host would need to go through that to boot an Ubuntu RISC-V domain.
I got told that any u-boot build would work. Didn't have the opportunity to test
it.
I also believe that there's nothing particularly different in how RISC-V works to
require a -kernel argument every time. Users don't have to do that with x86 and
even ppc64 pSeries domains: you give a disk with the guest image and everything
just works. Ideally we would want that for RISC-V domains as well. We'll probably
need stuff done in QEMU and OpenSBI to support this use case but I believe it's
worth it.
Depending on how much work/how long it would take we can discuss how libvirt can
ease the users pain in the meantime.
>
> With some relatively straightforward plumbing on the libvirt side, we
> should be able to use the information provided in that file to make
> the simple VM configuration
>
> <os firmware='u-boot'>
>
> work out of the box.
>
> Getting the Fedora version to work would be trickier. As far as I can
> tell, there's currently no spec-compliant way to describe a firmware
> that requires both -bios and -kernel to be used at the same time.
> Should the spec be extended? Can we get Fedora to standardize on the,
> at least to an outside observer, simpler approach that Ubuntu has
> adopted?
I believe this has to be answered by the Fedora folks, but yeah, it would be nice
if Fedora could adopt the same approach that Ubuntu, Debian, OpenSuse and others
are using.
Thanks,
Daniel
>
>
> [1] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_with_libvirt
> [2] https://fedoraproject.org/wiki/Architectures/RISC-V/Installing#Boot_under_QEMU
> [3] https://wiki.ubuntu.com/RISC-V#Booting_with_QEMU
More information about the libvir-list
mailing list