[libvirt] Some questions about the libvirt ESX driver

Matthias Bolte matthias.bolte at googlemail.com
Wed Dec 30 20:29:10 UTC 2009

2009/12/30 Richard W.M. Jones <rjones at redhat.com>:
> I'm playing with VMWare ESX 4.0 and I decided to try driving it from
> libvirt [git version].  Here are some random questions and issues I
> encountered, in no particular order.
> Listing domains works fine, but:
> (a) I have to specify ?no_verify=1 to get it to ignore the lack of
> valid https certificate.  Is there some documentation on how to set up
> certificates correctly?  [vSphere gives exactly the same error]

The default ESX certificates are autogenerated and self-signed ones.
The 'ESX Configuration Guide' [1] suggest to replace them by CA-signed

The 'ESX Configuration Guide' [1] contains a section about 'Replace a
Default Certificate with a CA-Signed Certificate'.

I created new self-signed certificates as described on the libvirt
website [2] and copied them to the ESX server

  servercert.pem -> /etc/vmware/ssl/rui.crt
  serverkey.pem -> /etc/vmware/ssl/rui.key

Then restart the hostd process either with

  service mgmt-vmware restart

or if that doesn't work (like with my ESXi 3.5 on USB flash drive) use

  /etc/init.d/hostd restart

On my Ubuntu box I copied the cacert.pem to
/usr/share/ca-certificates/esx-certs/cacert.pem, appended
esx-certs/cacert.pem to /etc/ca-certificates.conf and executed
update-ca-certificates. After this libcurl can verify the ESX server

libvirt uses explicit paths for its certificates for a remote TLS
connection using gnutls. This is possible with libcurl too, using the
CURLOPT_CAINFO option. For example the ESX driver could use
LIBVIRT_CACERT as the remote driver does, but this would require the
ESX server certificates to be signed by the same CA as the libvirtd
certificates, because libcurl doesn't fall back to the system CA
certificates if verification with the explicitly specified CA fails.

> (b) I have to type the root password (of the ESX server) each time.
> Can I avoid that?
> $ sudo ./tools/virsh -c 'esx://' list --all
> Enter username for [root]:
> Enter root password for
>  Id Name                 State
> ----------------------------------
>  - TestLinux            shut off
>  - TestWin              shut off

The ESX driver needs to login to the ESX server. virsh uses the
default auth callback that asks you to type in the credentials (see
virConnectAuthCallbackDefault in libvirt.c). Currently the only option
for automatic handling is policykit, but that doesn't work for the ESX
driver because policykit is not meant to provide credentials.

A possible way to "automatically" provide the credentials is to pass
them in the connection URI like this: esx://<user>:<pass>@hostname.
But this is currently not implemented in the ESX driver.

Another option would be to add new automatic authentication methods to
virConnectAuthCallbackDefault based on VIR_CRED_EXTERNAL and to change
the ESX driver to use them.

> (c) Starting domains doesn't seem to work, and the error message is
> obscure:
> $ sudo ./tools/virsh -c 'esx://' start TestLinux
> Enter username for [root]:
> Enter root password for
> error: Failed to start domain TestLinux
> error: internal error HTTP response code 500 for call to 'PowerOnVM_Task'. Fault: ServerFaultCode - The operation is not allowed in the current state.
> Is this to do with the "ask questions" API that was discussed on this
> list before?  Specifying auto_answer=0 or auto_answer=1 didn't make
> any difference.

No, the error message would say that there is a question blocking the
task. This message comes directly from the ESX server, as part of the
500er response.

I've never seen this error message from the ESX server before. Many
errors reported by the ESX server contain a details section, but the
deserialization of this details requires many new SOAP types and I
haven't implemented this yet.

Can you start the domain using the VI client? Maybe the domain really
is in a state where it cannot be started.

Can you reproduce this error with other domains?

> (d) dumpxml XML output looks very good, and consistent with the other
> drivers.  However it produces an odd <source file> attribute:
>    <disk type='file' device='disk'>
>      <driver name='lsilogic'/>
>      <source file='[Storage1] TestLinux/TestLinux.vmdk'/>
>      <target dev='sda' bus='scsi'/>
>    </disk>
>    <disk type='file' device='cdrom'>
>      <source file='[Storage1] Fedora-12-x86_64-Live.iso'/>
>      <target dev='hdc' bus='ide'/>
>    </disk>

What's wrong with it in your opinion? The <source file> attribute is
correct. See the ESX driver website section [3] about this.

> (e) pool-list, net-list are not supported by the driver.
> That's a particular problem for me because I was really hoping to use
> the storage APIs from libvirt to access the VMDK files that contain
> domains (eg. [Storage1] TestLinux/TestLinux.vmdk in the example
> above).  Is support for storage (particularly) and/or networks on the
> horizon for this driver?
> Rich.

I plan to implement the ESX driver as complete as possible, this
includes network and storage handling. I've took a look at this some
time ago and saw some problems. For examples datastores don't have a
UUID, so I may need to make them up and store them somewhere like the
IBM Power driver does it for the domain UUIDs, the libvirt network
model cannot represent a vSwitch ... I think these drivers are much
harder to implemented than the main ESX driver, so don't expect any
news on this in the next weeks, but stay tuned :)

What do you mean by "to access the VMDK files that contain domains",
just list them?

What part of the storage driver would that involve?

[1] http://www.vmware.com/pdf/vsphere4/r40/vsp_40_esx_server_config.pdf
[2] http://www.libvirt.org/remote.html#Remote_certificates
[3] http://www.libvirt.org/drvesx.html#xmlspecial


More information about the libvir-list mailing list