[RFC 00/21] RFC: Generate parsexml/formatbuf functions based on directives

Shi Lei shi_lei at massclouds.com
Wed Jun 10 01:20:28 UTC 2020

Last RFC: [https://www.redhat.com/archives/libvir-list/2020-April/msg00970.html]
In last RFC, I suggested we can generate object-model code based on relax-ng
files and Daniel gave it some comments.

Follow the suggestion from Daniel, I make another try to generate parsexml/formatbuf
functions automatically.


Still, we need several directives that can direct a tool to generate functions.
These directives work on the declarations of structs. For example:

    typedef struct _virNetworkDNSTxtDef virNetworkDNSTxtDef;
    typedef virNetworkDNSTxtDef *virNetworkDNSTxtDefPtr;
    struct _virNetworkDNSTxtDef {   /* genparse:concisehook, genformat */
        char *name;                 /* xmlattr, required */
        char *value;                /* xmlattr */

    typedef struct _virNetworkDNSSrvDef virNetworkDNSSrvDef;
    typedef virNetworkDNSSrvDef *virNetworkDNSSrvDefPtr;
    struct _virNetworkDNSSrvDef {   /* genparse:withhook, genformat */
        char *service;              /* xmlattr */
        char *protocol;             /* xmlattr */
        char *domain;               /* xmlattr */
        char *target;               /* xmlattr */
        unsigned int port;          /* xmlattr */
        unsigned int priority;      /* xmlattr */
        unsigned int weight;        /* xmlattr */

    typedef struct _virNetworkDNSHostDef virNetworkDNSHostDef;
    typedef virNetworkDNSHostDef *virNetworkDNSHostDefPtr;
    struct _virNetworkDNSHostDef {  /* genparse:withhook, genformat */
        virSocketAddr ip;           /* xmlattr */
        size_t nnames;
        char **names;               /* xmlelem:hostname, array */

    typedef struct _virNetworkDNSForwarder virNetworkDNSForwarder;
    typedef virNetworkDNSForwarder *virNetworkDNSForwarderPtr;
    struct _virNetworkDNSForwarder {    /* genparse:withhook, genformat */
        char *domain;                   /* xmlattr */
        virSocketAddr addr;             /* xmlattr */

    typedef struct _virNetworkDNSDef virNetworkDNSDef;
    typedef virNetworkDNSDef *virNetworkDNSDefPtr;
    struct _virNetworkDNSDef {              /* genparse:withhook, genformat */
        virTristateBool enable;                 /* xmlattr */
        virTristateBool forwardPlainNames;      /* xmlattr */
        size_t nforwarders;
        virNetworkDNSForwarderPtr forwarders;   /* xmlelem, array */
        size_t ntxts;
        virNetworkDNSTxtDefPtr txts;            /* xmlelem, array */
        size_t nsrvs;
        virNetworkDNSSrvDefPtr srvs;            /* xmlelem, array */
        size_t nhosts;
        virNetworkDNSHostDefPtr hosts;          /* xmlelem, array */

Explanation for these directives:

    - genparse[:withhook|concisehook]

      Only work on a struct.
      Generate parsexml function for this struct only if 'genparse' is specified.
      The function name is based on the struct-name and suffixed with 'ParseXML'.
      E.g., for struct virNetworkDNSDef, its parsexml function is

      If a parsexml function includes some error-checking code, it needs a
      post-process hook to hold them. Specify 'withhook' or 'concisehook' to
      setup a hook point in the parsexml function.

      Executing the tool manually can show the declaration of hook function.
      E.g. check declaration of 'virNetworkDNSDefParseXMLHook'.

        # ./build-aux/generator/go show virNetworkDNSDef -kp

        virNetworkDNSDefParseXMLHook(xmlNodePtr node,
                                     virNetworkDNSDefPtr def,
                                     const char *instname,
                                     void *opaque,
                                     const char *enableStr,
                                     const char *forwardPlainNamesStr,
                                     int nForwarderNodes,
                                     int nTxtNodes,
                                     int nSrvNodes,
                                     int nHostNodes);

      Some helper arguments (such as 'enableStr', 'nTxtNodes', etc.) are
      passed in to indicate the existence of fields.
      If these arguments are useless, specify 'concisehook' to omit them.

      When 'genparse' is specified, clear function for this struct is also
      generated implicitly, it is because the generated parsexml function needs
      to call the clear function.

    - genformat

      Only work on a struct.
      Generate formatbuf function for this struct only if 'genformat' is specified.
      The function name is based on struct-name and suffixed with 'FormatBuf'.

    - xmlattr[:thename]

      Parse/Format the field as an XML attribute.
      If 'thename' is specified, use it as the XML attribute name;
      or use the filed name.

    - xmlelem[:thename]
      Parse/Format the field as a child struct.
      If 'thename' is specified, use it as the XML element name;
      or use the filed name.

    - array

      Parse/Format the field as an array.
      Its related field is a counter, which name should follow the pattern:
      n + 'field_name'.

    - required

      The field must have a corresponding item defined in XML.


    1. If a field isn't specified with 'xmlattr' or 'xmlelem', it will be
       ignored in the parsexml/formatbuf functions.

    2. For enum, use its name rather than int.
       E.g., the type of the field 'enable' in virNetworkDef should be
       'virTristateBool' rather than 'int'.


The Tool is based on libclang and its python-binding.

It has three subcommands: 'list', 'show' and 'generate'.
The 'list' and 'show' are used for previewing generated code;
the 'generate' is prepared to be invoked by Makefile whenever the c header files
under 'src/conf' and 'src/util' have changed.

About this series

To generate all parsexml/formatbuf functions for virNetworkDef and all its
subordinate structs, it needs around 95 patches.

In this RFC, I just post 21 patches for evaluation.


Shi Lei (21):
  build-aux: Add a tool to generate xml parse/format functions
  maint: Check libclang and its python3 binding
  maint: Call generator automatically when c-head-files change
  maint: Add helper macro VIR_USED
  util: Add two helper functions virXMLChildNode and virXMLChildNodeSet
  conf: Extract error-checking code from virNetworkDNSTxtDefParseXML
  conf: Replace virNetworkDNSTxtDefParseXML(hardcoded) with
  conf: Generate virNetworkDNSTxtDefFormatBuf
  conf: Extract error-checking code from virNetworkDNSSrvDefParseXML
  conf: Replace virNetworkDNSSrvDefParseXML(hardcoded) with
  conf: Generate virNetworkDNSSrvDefFormatBuf
  util: Add parsexml/formatbuf helper functions for virSocketAddr
  conf: Extract error-checking code from virNetworkDNSHostDefParseXML
  conf: Replace virNetworkDNSHostDefParseXML(hardcoded) with
  conf: Generate virNetworkDNSHostDefFormatBuf
  conf: Extract virNetworkDNSForwarderParseXML from
  conf: Replace virNetworkDNSForwarderParseXML(hardcoded) with
  conf: Generate virNetworkDNSForwarderFormatBuf
  conf: Extract error-checking code from virNetworkDNSDefParseXML
  conf: Replace virNetworkDNSDefParseXML(hardcoded) with
  conf: Generate virNetworkDNSDefFormatBuf

 build-aux/generator/directive.py | 839 +++++++++++++++++++++++++++++++
 build-aux/generator/go           |  14 +
 build-aux/generator/main.py      | 416 +++++++++++++++
 build-aux/generator/utils.py     | 100 ++++
 configure.ac                     |  12 +
 po/POTFILES.in                   |   2 +
 src/Makefile.am                  |  15 +
 src/access/Makefile.inc.am       |   2 +-
 src/conf/Makefile.inc.am         |  13 +-
 src/conf/network_conf.c          | 487 ++++--------------
 src/conf/network_conf.h          |  54 +-
 src/esx/Makefile.inc.am          |   2 +-
 src/interface/Makefile.inc.am    |   2 +-
 src/internal.h                   |   2 +
 src/lxc/Makefile.inc.am          |   1 +
 src/network/Makefile.inc.am      |   2 +-
 src/network/bridge_driver.c      |   2 +-
 src/node_device/Makefile.inc.am  |   2 +-
 src/nwfilter/Makefile.inc.am     |   2 +-
 src/qemu/Makefile.inc.am         |   1 +
 src/remote/Makefile.inc.am       |   2 +-
 src/secret/Makefile.inc.am       |   2 +-
 src/storage/Makefile.inc.am      |   2 +-
 src/test/Makefile.inc.am         |   2 +-
 src/util/Makefile.inc.am         |  12 +-
 src/util/virsocketaddr.c         |  38 ++
 src/util/virsocketaddr.h         |  22 +-
 src/util/virxml.c                |  57 +++
 src/util/virxml.h                |   3 +
 src/vbox/Makefile.inc.am         |   2 +-
 tests/Makefile.am                |   2 +
 tools/Makefile.am                |   2 +
 32 files changed, 1674 insertions(+), 442 deletions(-)
 create mode 100644 build-aux/generator/directive.py
 create mode 100755 build-aux/generator/go
 create mode 100755 build-aux/generator/main.py
 create mode 100644 build-aux/generator/utils.py


More information about the libvir-list mailing list