[virt-tools-list] Re: libosinfo - another try

[Cole Robinson]

On 10/22/2009 05:47 AM, Arjun Roy wrote:

cc-ing a few other folks (rjones, mdbooth)

> Hi all,
> 
> I know there's been some discussion in months past for replacing the OS metadata
> currently stored as a python dict in virtinst, so it can be used by various
> virt applications.
> 

Thanks for picking this up! Some comments inline:

> I refer to Cole Robinson's initial draft of an API called libosinfo linked here:
> 
> http://www.mail-archive.com/et-mgmt-tools@redhat.com/msg02673.html
> 
> While there was a significant amount of discussion on it, there doesn't seem to
> be any progress made on the front so I thought I'd present a refined API for
> consideration.
> 
> Unlike Cole, I haven't done any implementation or figured out any implementation
> details at all. I'm just throwing some ideas out there, based on what the
> previous discussion came up with:
> 
> The basic idea would be as follows:
> 
> 1. All the data stored right now as a python dict would be replaced with a
> collection of xml (or any other kind of markup that works) file encoding the
> same data.
> 2. The data would *not* contain a fixed hierarchy - ie. we wouldn't have a tag
> for vendor followed by a tag for distro, etc. Instead, the data would consist of
> a flat set of distros, with several properties such as vendor or version (etc.)
> that would allow an application using the API to impose any arbitrary sort of
> hierarchy on the data.
> 
> What follows is my take on what form the API and data representation scheme
> could look like:
> 
> ________________________________________
> 
> Data Representation Schema
> ________________________________________
> 
> The primary storage record would be the <distro>. It would consist of a required
> ID, and one or more optional attributes.
> 
> There would be two kinds of attributes:
> 
> 1. Structure Attributes
> 
> These attributes would encode possible relationships between distros. Proposed
> attributes would be:
> <derives-from>
> <clones>
> <upgrades>
> 
> So one could have the following:
> <distro ID=1>
>   <name>Fedora 10</name>
> </distro>
> 
> <distro ID=2>
>   <name>Fedora 11</name>
>   <upgrades ID=1 />
> </distro>
> 

If we are going to use 'ID' here as the unique identifier, I think it
should have some human readable value. Just using numbers will quickly
become confusing if manually navigating the XML.

In the previous discussion, someone mentioned just using <name> as the
unique identifier, however I don't think that is sufficient either,
since it is not very machine friendly. Something like

"Fedora 12" -> "fedora12"
"Red Hat Enterprise Linux 5.4" -> "rhel5.4" or
                                  "red_hat_enterprise_linux_5.4"

Think of using this value from the command line (like virt-install
--os-variant), no one is going to want to have to use spaces and proper
capitalization, or a plain digit.

> This would effectively force our view of the data as a graph, where the set of
> nodes are the set of distros and the edges are described by the structure
> attribute tags. (With a different graph resulting depending on attribute)
> 
> 2. Informative Attributes
> This could refer to things such as name, kernel-type, architecture, or anything
> else that would be useful to track. I suspect that it would be alright to limit
> the kind of data to strings (kernel : linux) and version numbers 
> (kernel-version: 2.6.30).
> 

I kind of wonder if the 'version' distinction is even necessary, but I
don't think I could say for certain until we start trying to use the code.

> In other words, we'd end up with a schema like this:
> <distro ID=2>
>   <upgrades ID=1 />
>   <name type=str>Fedora 11</name>
>   <kernel type=str>linux</kernel>
>   <kernel-version type=ver>2.6.30</kernel-version>
> </distro>
> 
> ________________________________________
> 
> API
> ________________________________________
> 
> 
> The library would be implemented in C, and define a few major types:
> 
> osi_lib_t : an opaque handle to a library instance.
> osi_distro_t : represents a distro object.
> osi_distro_id_t : represents the unique ID for a distro.
> osi_distro_list_t : a list of osi_distro_t objects.
> osi_filter_t: filter for searching through distros.
> 
> And a bunch of methods:
> 
> osi_lib_t osi_getLibHandle();
> 
> /* Setting parameters like libvirt version, etc. */
> int osi_setLibParam(osi_lib_t lib, cstring key, cstring val);
> int osi_getLibParam(osi_lib_t lib, cstring key);
> 
> /* Initializing and closing down the library */
> int osi_initLib(osi_lib_t lib);
> int osi_closeLib(osi_lib_t lib);
> 
> /* Querying for distros and iterating over them */
> osi_distro_list_t osi_getDistrosList(osi_lib_t lib, osi_distro_filter_t filter);
> int osi_putDistrosList(osi_distro_list_t distros);
> int osi_distroListLength(osi_distro_list_t distros);
> 
> /* Methods for filtering results when querying distros */
> osi_filter_t osi_getFilter(osi_lib_t lib);
> int osi_putFilter(osi_filter_t filter);
> int osi_addStringPropertyConstraint(osi_filter_t filter, cstring propName, cstring propVal);
> int osi_addVersionPropertyConstraint(osi_filter_t filter, cstring propName, cstring propVal, enum_t ordering);
> int osi_addLinkedDistroConstraint(osi_filter_t filter, enum_t relationship, os_distro_t distro);
> int osi_clearStringPropertyConstraint(osi_filter_t filter, cstring propName);
> int osi_clearVersionPropertyConstraint(osi_filter_t filter, cstring propName);
> int osi_clearLinkedDistroConstraint(osi_filter_t filter, enum_t relationship);

I think all these calls mean we should break out an 'osi_prop_t'. This
way, we only have one call for addProp, clearProp, getProp, and then a
set of methods to manipulate properties.

Not sure how the implementation details would work out.

> int osi_clearAllConstraints(osi_filter_t filter);
> 
> /* Get a single distro, either from a list or by ID */
> osi_distro_t osi_getDistroByIndex(osi_distro_list_t distros, int index);
> osi_distro_t osi_getDistroById(osi_lib_t lib, osi_distro_id_t id);
> 
> /* Query Properties for a given distro */
> osi_distro_id_t osi_getDistroId(osi_distro_t distro);
> osi_distro_t osi_getLinkedDistro(osi_distro_t distro, enum_t relationship);
> cstring osi_getStringProperty(osi_distro_t distro, cstring propName);
> cstring osi_getVersionProperty(osi_distro_t distro, cstring propName);
> cstring** osi_getAllStringProperties(osi_distro_t distro);
> cstring** osi_getAllVersionProperties(osi_distro_t distro);
> 
> /* Query unique values for a given property */
> cstring *osi_uniqueStringPropertyValues(osi_lib_t lib, cstring propName);
> cstring *osi_uniqueVersionPropertyValues(osi_lib_t lib, cstring propName);
> os_distro_list_t osi_uniqueLinkedDistroValues(osi_lib_t lib, enum_t relationship);
> 
> The general usage is as follows. 
> -We get a library handle, set some parameters like libvirt version and 
> hypervisor type, and initialize it. 
> -We can then get a filter object, configure it to search for all distros with a 
> 'linux' kernel attribute that have kernel version < 2.6.30 that derive from 
> RHEL 5.4.
> -Then we can query other properties, such as the preferred audio
> driver (the answer for which might change based on the hypervisor we specified
> at library init, for example).
> -Then we can query the unique values for the property 'Vendor' to get a list of 
> strings representing vendor names. 
> -Or, we could query unique values for the relationship property DERIVES_FROM to
> get a list of distros that are derived by at least one other distro.
> -At some point we're done looking up data and close the library handle.
> 
> I look forward to any feedback.
> 

Overall it looks good, there seems to be enough here to handle even the
difficult cases

I think some (psuedo)code examples using the API would also go a long
way in helping reviewers wrap their head around the code: it could also
expose some awkwardness in the above methods.

Thanks,
Cole