[libvirt] Building on Solaris 11 Express

[Matthias Bolte]

2011/6/4 Ruben Kerkhof <ruben at rubenkerkhof.com>:
> On Sat, Jun 4, 2011 at 10:40, Matthias Bolte
> <matthias.bolte at googlemail.com> wrote:
>> Okay VirtualBox 4.0.8 work with libvirt on Windows too. So this seems
>> to be a Solaris specific problem. Does an earlier version of
>> VirtualBox work, for example 4.0.0 or 3.2.10, or did it never work?
>
> I just tried 4.0.0, same behaviour.
> 3.2.12 does:
>
> ruben at openindiana:~$ virsh list
> VBoxSVC: error: 0 bytes read from child process
> error: Failed to reconnect to the hypervisor
> error: no valid connection
> error: internal error nsIEventQueue object is null
>
> This is with no running vms, only one which is stopped.
> I could have a look with gdb, if you can tell me which function I have
> to set a breakpoint on.
>
> Thanks!
>
> Ruben
>

Try running with full debug and logging

LIBVIRT_DEBUG=1 LIBVIRT_LOG_OUTPUTS=1:stderr virsh list

and post the output here. I want to verify that the rest is okay and
the problem is there where I expect it.

What the VirtualBox driver does is dlopen the VBoxXPCOMC.so, get the
entry function VBoxGetXPCOMCFunctions and call it, see tryLoadOne in
src/vbox/vbox_XPCOMGlue.c. This seems to work otherwise it wouldn't
have gotten to the nsIEventQueue error. Then in src/vbox/vbox_tmp.c in
vboxInitialize pfnComInitialize is called to initialize the XPCOM
interface and get the IVirtualBox and ISession objects. After that it
tries to obtain the IEventQueue object and fails because it's NULL.
The error reporting order in this function is suboptimal and I'll fix
that. Actually it should check the IVirtualBox and ISession object for
NULL first because the call to pfnComInitialize probably already
failed. But the signature of pfnComInitialize is bad, it should return
a value to indicate an error. You also can't tell what the actual
problem is because pfnComInitialize internally logs the reason but
logging is typically compiled out.

pfnComInitialize points to VBoxComInitialize in
src/VBox/Main/cbindings/VBoxXPCOMC.cpp in the VirtualBox source code.
Last time I debugged such a problem I ended up compiling VBoxXPCOMC.so
from source and replace the system VBoxXPCOMC.so with my own one that
just wrote debug messages to a file in /tmp so that I could figure out
the actual problem. Last time it was a mismatch in the COM object IDs
for the IVirtualBox and ISession objects. libvirt accidentally used
the IDs from a beta VirtualBox SDK and the IDs had changed in the
actual release. This made the CreateInstanceByContractID calls in
VBoxComInitialize fail, and VBoxComUninitialize is called that
releases the IEventQueue object and it's NULL when the VirtualBox
driver checks it later. So the error message about IEventQueue being
NULL is just a symptom of VBoxComInitialize failing.

If we want to fix this we need to figure out why VBoxComInitialize
fails. You could try to break on VBoxComInitialize in VBoxXPCOMC.so in
gdb. You could also try to make gdb step into the call to
VBoxComInitialize in vboxInitialize via pfnComInitialize. I fail at
this, maybe it's not even possible. As said last time I compiled my
own VBoxXPCOMC.so with debug log output in it to figure out the
problem.

Matthias