[libvirt] [PATCHv5 5/6] bhyve: Add console support for grub-bhyve bootloader

[Conrad Meyer]

On Tue, Oct 28, 2014 at 7:53 AM, Roman Bogorodskiy
<bogorodskiy at gmail.com> wrote:
>   Conrad Meyer wrote:
>
>> This enables booting interactive GRUB menus (e.g. install CDs) with
>> libvirt-bhyve.
>>
>> Caveat: A terminal other than the '--console' option to 'virsh start'
>> (e.g. 'cu -l /dev/nmdm0B -s 115200') must be used to connect to
>> grub-bhyve because the bhyve loader path is synchronous and must occur
>> before the VM actually starts.
>
> Could you please elaborate on that?

Sure.

> It's not the obvious what's the
> limitation...

The limitation is in virBhyveProcessStart and bhyveload / bhyve
itself. The problem is that we cannot start bhyve until the loader has
exited. So in BhyveProcessStart, we run the loader synchronously, then
run bhyve asynchronously when the loader completes. Finally when
BhyveProcessStart returns and sets the domain as started, 'virsh
console' and 'virsh start --console' connect and start working.

To allow 'virsh start --console', etc, I believe we'd have to start
bhyveload / GRUB asynchronously in BhyveProcessStart, mark the domain
as started, handle the asynchronous termination of bhyveload / GRUB,
start bhyve... it sounds very painful.

I suppose it would be a lot easier / cleaner if the bhyve binary
itself learned how to start the bootloader (even if we had to
explicitly give it the executable and arguments).

> Without being able to connect using virsh console, it's
> not very convenient for user to connect to the console (esp. if he runs
> virsh over a remote libvirt).

Agreed. It's not perfect, but is an incremental improvement. I think
probably most VMs should be configured to boot without getting stuck
in the bootloader.

>
>> Changing the bhyveProcessStart logic around to accommodate '--console'
>> for interactive loader use seems like a significant project and probably
>> not worth it, if UEFI/BIOS support for bhyve is "coming soon."
>>
>> Sponsored by:  EMC / Isilon storage division
>>
>> Signed-off-by: Conrad Meyer <conrad.meyer at isilon.com>
>> ---
>>  src/bhyve/bhyve_command.c | 15 +++++++++++++++
>>  1 file changed, 15 insertions(+)
>>
>> diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c
>> index 01f1795..07d209e 100644
>> --- a/src/bhyve/bhyve_command.c
>> +++ b/src/bhyve/bhyve_command.c
>> @@ -468,6 +468,21 @@ virBhyveProcessBuildGrubbhyveCmd(virDomainDefPtr def,
>>      virCommandAddArgFormat(cmd, "%llu",
>>                             VIR_DIV_UP(def->mem.max_balloon, 1024));
>>
>> +    if (def->nserials > 0) {
>> +        virDomainChrDefPtr chr;
>> +
>> +        chr = def->serials[0];
>> +
>> +        if (chr->source.type != VIR_DOMAIN_CHR_TYPE_NMDM) {
>> +            virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
>> +                           _("only nmdm console types are supported"));
>> +            return NULL;
>> +        }
>> +
>> +        virCommandAddArg(cmd, "--cons-dev");
>
> IMHO, one more thing worth to do is probe if the supplied grub-bhyve has
> this '--cons-dev' argument (e.g. check if it's there in its --help
> output). Otherwise, for users running an older grub-bhyve will get a
> weird error that they will not know how to fix.

I think there are few enough bhyve-grub and libvirt users at this
point, and the experience is already broken enough, that we can just
document in release notes that libvirt-1.x.y Bhyve support needs
grub2-bhyve >= 808fa4f1b1 and ensure packagers and porters know to get
the run-time dependency correct.

> Other option is to check required version of grub-bhyve using autotools,
> but I guess it's not good because users will have to rebuild libvirt if
> they want to update bhyve-grub.

Hm, why would they have to rebuild?

Thanks,
Conrad