[Libvir] Proposal for the storage API (for discussion only)

[Richard W.M. Jones]

Daniel P. Berrange wrote:
> On Fri, Oct 19, 2007 at 09:53:27AM +0100, Richard W.M. Jones wrote:
>> Daniel P. Berrange wrote:
>>> Using structures in the public API is not in keeping with the rest of
>>> the libvirt APIs. We should be using XML for the main metadata description
>>> of volumes & pools.
>> No, that doesn't make sense.  XML for an API is a hack.  It's hard to 
>> use it without requiring an additional external library to parse the 
>> XML.  It's slow.  It has the facade of maintaining ABI compatibility 
>> (because it's "just strings"), but in fact has no guaranteed ABI at all. 
>>  It sits very poorly with static typing, virtually guaranteeing runtime 
>> errors that you are forced to do something about.
> 
> The speed is irrelevant in the context we're using this. Neither creating
> nor getting metadata about storage volumes are performance critical methods.
> Parsing XML is linenoise when creating a volume in comparison to the actual
> time required to allocate the volume itself.

So speed doesn't matter.  What about my other points?

XML requires extra libraries and efforts to parse something that simply 
does not need to be parsed.  It could be presented directly as structures.

XML sits poorly with static typing.

XML introduces possible run-time errors that have to be dealt with by 
callers.  (eg. Basic fields which you expect to be there, but for some 
reason are not).

>> See previous discussion about capabilities, also a hack:
>> https://www.redhat.com/archives/libvir-list/2007-March/msg00104.html
> 
> The use of XML for describing metadata about managed objects is a core
> aspect of libvirt. We can debate use of XML vs structures for hours on
> end, but the fact remains that libvirt uses XML for its metadata APIs.

XML is used in a few places: As a way to describe domains (in & out) and 
for capabilities (out).  The original capabilities proposal didn't use 
XML -- I'm not proud that I folded on that one. 
virConnectGetCapabilities returns sprawling XML that differs in ways we 
don't really understand between hypervisors.  We have avoided defining 
real meanings to much of the capabilities XML.  This is not a good 
example to follow.

If this is going to be decided by some sort of popularity contest 
(`libvirt uses XML already so why not use a bit more'), then how about 
we look at other comparable C libraries.  How many of them return XML 
from API calls, that is when they're not returning HTML documents or the 
like?

>>> This is just one reason why I don't like the idea of simply running shell
>>> scripts in the back end. This will result in a unreliable, hard-to debug
>>> system. It may be a short term win on implementation speed, but it will be
>>> a PITA for long term maintainence. One thing you will immediately get is 
>>> people writing scripts which add all kinda of custom crap to the XML 
>>> destroying the benefit of libvirt which is the standardization. For any
>> I don't understand this.  Scripts can be written which add custom fields 
>> to the XML, but since libvirt will just ignore those fields I don't see 
>> any issue.
>>
>>> given storage backend, we know what operations we need to be able to 
>>> perform & so we have a finite set of commands we need to run with known
>>> predictable arguments.  We just don't need the 'flexibility' of running
>>> arbitrary shell scripts & XML filters in the backend end.
>> We do because we need to be able to take the output of 'vgs', 'sfdisk', 
>> 'iscsiadm', etc., the output of each being essentially the same 
>> information (a list of volume groups plus the size and free space of 
>> each), and present that information in the same format back to libvirt. 
>>  In this case a common format makes perfect sense.  It need not be XML, 
>> it might be CSV, but in this case XML's extensibility makes sense, along 
>> with the fact that libvirt already parses XML.
> 
> This is just pushing the parsing code out of libvirt and into an external
> application. I am saying that libvirt should deal with it directly, since
> this is the core job of the driver APIs. libvirt should call 'vgs' directly
> and parse its output directly. Adding extra layers in the stack which
> transform vgs output into some further intermediate form is not going to
> make this easy to debug.

I think this is fair enough.  The important part is to make sure that 
the sysadmin can configure it, and it doesn't make much difference 
whether we use scriptlets or just have configuration options.

Rich.

-- 
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom.  Registered in
England and Wales under Company Registration No. 03798903
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20071019/4cc9bc4a/attachment-0001.bin>