[virt-tools-list] [vhostmd virtio PATCH v3 5/6] Activate virtio support in vhostmd

Jim Fehlig jfehlig at suse.com
Wed Nov 14 23:46:13 UTC 2018


On 11/12/18 8:13 AM, Michael Trapp wrote:
> Activate the virtio code and extend DTD/XML to support a virtio section
> for virtio related configuration and a new transport type virtio.
> ---
>   README              | 84 +++++++++++++++++++++++++++++++++++++++++----
>   vhostmd.changes     |  5 +++
>   vhostmd.dtd         |  6 +++-
>   vhostmd.xml         |  6 ++++
>   vhostmd/Makefile.am |  4 +--
>   vhostmd/vhostmd.c   | 61 ++++++++++++++++++++++++++++++--
>   6 files changed, 154 insertions(+), 12 deletions(-)
> 
> diff --git a/README b/README
> index cb23226..f473079 100644
> --- a/README
> +++ b/README
> @@ -50,8 +50,15 @@ includes a few examples of user-defined metrics, which provide a
>           <path>/dev/shm/vhostmd0</path>
>           <size unit="k">256</size>
>         </disk>
> +      <virtio>
> +        <max_channels>1024</max_channels>
> +        <expiration_time>15</expiration_time>
> +        <path>/var/lib/libvirt/qemu/channels</path>

We should update this to /var/lib/libvirt/qemu/channel/target, depending on the 
answer to my question in 4/6.

> +      </virtio>
>         <update_period>5</update_period>
>         <path>/usr/bin:/usr/sbin:/usr/share/vhostmd/scripts</path>
> +      <transport>vbd</transport>
> +      <transport>virtio</transport>
>       </globals>
>       <metrics>
>         <metric type="string" context="host">
> @@ -115,11 +122,17 @@ includes a few examples of user-defined metrics, which provide a
>     </vhostmd>
>   
>   A valid configuration file must contain the root element <vhostmd>.
> -The <globals> element contains configuration global to vhostmd, such as
> -the metrics disk path and the metrics refresh interval.  The <metrics>
> -element is a container for all of the <metric> elements.  A metric element
> -is used to define a metric, giving it a name and an action that produces
> -the metric value.
> +
> +The <globals> element contains configuration global to vhostmd, such as the
> +metrics refresh interval and the metrics transport mechanism. The <transport>
> +element defines how the metrics are transported between the host and VMs. The
> +vbd transport uses a virtual disk, described in the <disk> element, to share
> +metrics data between host and VM. The virtio transport, described by the
> +<virtio> element, uses a virtio-serial connection to share the metrics data.
> +
> +The <metrics> element is a container for all of the <metric> elements.
> +A metric element is used to define a metric, giving it a name and an action
> +that produces the metric value.
>   
>   The supplied vhostmd configuration file provides a useful set of default
>   metrics to be collected.  This can be extended or modified by editing
> @@ -277,14 +290,71 @@ section:
>   (Note: Change target dev and bus as appropriate for the domain)
>   
>   
> +
> +Notes on Virtio Transport
> +-------------------------
> +
> +The virtio transport uses a virtio serial device to transport metrics data
> +between the host and VMs. Basically for a virtio serial device, QEMU creates
> +- a unix domain socket on the Host
> +- a serial port on the VM
> +- 'connects' both to a 'communication channel'
> +
> +It can be configured in the virtio section of the vhostmd configuration file.
> +<max_channels> defines the maximum number of virtio channels/VMs supported
> +by the vhostmd instance with a default value of 1024.
> +<expiration_time> is the time after which the virtio serial channel of a VM
> +is detached when vhostmd did not receive updates. It's default value is
> +'3 * update_period' of the <globals> section and the configured value is
> +checked to be at least '3 * update_period'.
> +<path> is the directory where the virtio serial channels are accessible and
> +it must be the directory of the path attribute of QEMUs VM/channel
> +configuration. The default path is '/var/lib/libvirt/qemu/channels'.
> +
> +  <vhostmd>
> +    <globals>
> +      <virtio>
> +        <max_channels>1024</max_channels>
> +        <expiration_time>15</expiration_time>
> +        <path>/var/lib/libvirt/qemu/channels</path>

Same here.

> +      </virtio>
> +
> +Sample VM config with virtio serial:
> +
> +  <channel type='unix'>
> +    <source mode='bind' path='/var/lib/libvirt/qemu/channels/\
> +org.github.vhostmd.1.a70605c8-7d69-8c44-7e1a-5ecd092cb1e1'/>

Ideally we'd like to leave out the path and let libvirt generate it.

> +    <target type='virtio' name='vhostmd'/>

Here is where we use the reverse domain name and protocol version. E.g.

   <target type='virtio' name='org.github.vhostmd.1'/>

> +    <address type='virtio-serial' controller='0' bus='0' port='1'/>
> +  </channel>
> +
> +The name of a channel in a QEMU VM configuration must be
> +  <path from vhostmd-globals-virtio-path>/org.github.vhostmd.1.<VM-UUID>
> +
> +
> +Vhostmd accepts metric requests 'GET /metrics/XML\n\n' and responds with
> +
> +  <metrics>
> +    <metric type='real64' context='host'>
> +      <name>TotalCPUTime</name>
> +      <value>179645.910000</value>
> +    </metric>
> +  ...
> +    <metric type='uint64' context='vm' id='9' uuid='a70605c8-7d69-8c44-7e1a-5ecd092cb1e1'>
> +      <name>UsedMemory</name>
> +      <value>524288</value>
> +    </metric>
> +  </metrics>
> +
> +
>   Guest Tool/Library for Accessing Metrics Data
>   ---------------------------------------------
>   
>   Tool: vm_dump_metrics
>    Stand alone static utility will read all the metrics and write them
>    to stdout or optionally an argumented file.
> - Usaga:
> -   vhostmd [-f dest_file]
> + Usage:
> +   vm_dump_metrics -b|-i|-x [-d dest_file]
>   
>   Library: libmetrics.so.0
>    Dynamic library that supports individual metrics gathering
> diff --git a/vhostmd.changes b/vhostmd.changes
> index c7b453d..c1aa06f 100644
> --- a/vhostmd.changes
> +++ b/vhostmd.changes
> @@ -1,3 +1,8 @@
> +-------------------------------------------------------------------
> +Wed Jul 25 10:00:20 CEST 2018 - michael.trapp at sap.com
> +
> +- add virtio as transport mechanism
> +
>   -------------------------------------------------------------------
>   Mon Jun 29 16:42:39 MDT 2009 - jfehlig at novell.com
>   
> diff --git a/vhostmd.dtd b/vhostmd.dtd
> index 471c72f..6480532 100644
> --- a/vhostmd.dtd
> +++ b/vhostmd.dtd
> @@ -9,7 +9,7 @@ Virtual Host Metrics Daemon (vhostmd). Configuration file DTD
>   -->
>   
>   <!ELEMENT vhostmd (globals,metrics)>
> -<!ELEMENT globals (disk,update_period,path,transport+)>
> +<!ELEMENT globals (disk,virtio,update_period,path,transport+)>
>   
>   <!ELEMENT disk (name,path,size)>
>   <!ELEMENT name (#PCDATA)>
> @@ -20,6 +20,10 @@ Virtual Host Metrics Daemon (vhostmd). Configuration file DTD
>   <!ELEMENT update_period (#PCDATA)>
>   <!ELEMENT transport (#PCDATA)>
>   
> +<!ELEMENT virtio (max_channels,expiration_time,path)>
> +<!ELEMENT max_channels (#PCDATA)>
> +<!ELEMENT expiration_time (#PCDATA)>
> +
>   <!ELEMENT metrics (metric*)>
>   <!ELEMENT metric (name,action,variable*)>
>   <!ELEMENT action (#PCDATA)>
> diff --git a/vhostmd.xml b/vhostmd.xml
> index 9b048df..5400981 100644
> --- a/vhostmd.xml
> +++ b/vhostmd.xml
> @@ -33,9 +33,15 @@ the logical && operator must be replaced with "&&".
>           <path>/dev/shm/vhostmd0</path>
>           <size unit="k">256</size>
>         </disk>
> +      <virtio>
> +        <max_channels>1024</max_channels>
> +        <expiration_time>15</expiration_time>
> +        <path>/var/lib/libvirt/qemu/channels</path>

Another spot potentially needing an adjustment.

Regards,
Jim

> +      </virtio>
>         <update_period>5</update_period>
>         <path>/usr/sbin:/sbin:/usr/bin:/bin:/usr/share/vhostmd/scripts</path>
>         <transport>vbd</transport>
> +      <transport>virtio</transport>
>         <!-- <transport>xenstore</transport> -->
>       </globals>
>       <metrics>
> diff --git a/vhostmd/Makefile.am b/vhostmd/Makefile.am
> index 3585970..34a15e0 100644
> --- a/vhostmd/Makefile.am
> +++ b/vhostmd/Makefile.am
> @@ -3,9 +3,9 @@ INCLUDES = \
>       -I../include
>   
>   sbin_PROGRAMS = vhostmd
> -vhostmd_SOURCES = vhostmd.c util.c metric.c virt-util.c
> +vhostmd_SOURCES = vhostmd.c util.c metric.c virt-util.c virtio.c
>   vhostmd_CFLAGS = $(LIBXML_CFLAGS) $(LIBVIRT_CFLAGS)
> -vhostmd_LDADD = -lm $(LIBXML_LIBS) $(LIBVIRT_LIBS)
> +vhostmd_LDADD = -lm $(LIBXML_LIBS) $(LIBVIRT_LIBS) -lpthread
>   
>   if WITH_XENSTORE
>   vhostmd_SOURCES += xenstore-update.c
> diff --git a/vhostmd/vhostmd.c b/vhostmd/vhostmd.c
> index e7ec2fc..e13fa2d 100644
> --- a/vhostmd/vhostmd.c
> +++ b/vhostmd/vhostmd.c
> @@ -42,10 +42,11 @@
>   #include <time.h>
>   #include <libxml/parser.h>
>   #include <libxml/xpath.h>
> +#include <pthread.h>
>   
>   #include "util.h"
>   #include "metric.h"
> -
> +#include "virtio.h"
>   
>   /*
>    * vhostmd will periodically write metrics to a disk.  The metrics
> @@ -85,6 +86,7 @@ typedef struct _mdisk_header
>    */
>   #define VBD      (1 << 0)
>   #define XENSTORE (1 << 1)
> +#define VIRTIO   (1 << 2)
>   
>   /* Global variables */
>   static int down = 0;
> @@ -103,6 +105,9 @@ static mdisk_header md_header =
>            };
>   static char *search_path = NULL;
>   static int transports = 0;
> +static char *virtio_path = "/var/lib/libvirt/qemu/channels";
> +static int virtio_max_channels = 1024;
> +static int virtio_expiration_time = 15;
>   
>   
>   /**********************************************************************
> @@ -470,6 +475,8 @@ static int parse_transports(xmlDocPtr xml,
>   	     return -1;
>   #endif
>   	 }
> +         if (strncasecmp((char *)str, "virtio", strlen("virtio")) == 0)
> +             transports |= VIRTIO;
>            free(str);
>         }
>      }
> @@ -605,6 +612,19 @@ static int parse_config_file(const char *filename)
>         goto out;
>      }
>       
> +   if (transports & VIRTIO) {
> +      char *p = NULL;
> +
> +      if (vu_xpath_long("string(./globals/virtio/max_channels[1])", ctxt, &l) == 0)
> +         virtio_max_channels = (int)l;
> +
> +      if (vu_xpath_long("string(./globals/virtio/expiration_time[1])", ctxt, &l) == 0)
> +         virtio_expiration_time = (int)l;
> +
> +      if ((p = vu_xpath_string("string(./globals/virtio/path[1])", ctxt)) != NULL)
> +         virtio_path = p;
> +   }
> +
>      /* Parse requested metrics definitions */
>      if (parse_metrics(xml, ctxt)) {
>         vu_log(VHOSTMD_ERR, "Unable to parse metrics definition "
> @@ -838,7 +858,8 @@ static int metrics_disk_create(void)
>   static int metrics_host_get(vu_buffer *buf)
>   {
>      metric *m = metrics;
> -
> +   unsigned start = buf->use;
> +
>      while (m) {
>         if (m->ctx != METRIC_CONTEXT_HOST) {
>            m = m->next;
> @@ -850,12 +871,18 @@ static int metrics_host_get(vu_buffer *buf)
>            
>         m = m->next;
>      }
> +
> +   if (transports & VIRTIO)
> +      virtio_metrics_update(&buf->content[start], (int) (buf->use - start),
> +                            NULL, METRIC_CONTEXT_HOST);
> +
>      return 0;
>   }
>   
>   static int metrics_vm_get(vu_vm *vm, vu_buffer *buf)
>   {
>      metric *m = metrics;
> +   unsigned    start = buf->use;
>   
>      while (m) {
>         if (m->ctx != METRIC_CONTEXT_VM) {
> @@ -868,6 +895,11 @@ static int metrics_vm_get(vu_vm *vm, vu_buffer *buf)
>            
>         m = m->next;
>      }
> +
> +   if (transports & VIRTIO)
> +      virtio_metrics_update(&buf->content[start], (int) (buf->use - start),
> +                            vm->uuid, METRIC_CONTEXT_VM);
> +
>      return 0;
>   }
>   
> @@ -912,11 +944,30 @@ static int vhostmd_run(int diskfd)
>      int *ids = NULL;
>      int num_vms = 0;
>      vu_buffer *buf = NULL;
> +   pthread_t virtio_tid;
>      
>      if (vu_buffer_create(&buf, MDISK_SIZE_MIN - MDISK_HEADER_SIZE)) {
>         vu_log(VHOSTMD_ERR, "Unable to allocate memory");
>         return -1;
>      }
> +
> +   if (transports & VIRTIO) {
> +      int rc;
> +
> +      if (virtio_expiration_time < (update_period * 3))
> +         virtio_expiration_time = update_period * 3;
> +
> +      if (virtio_init(virtio_path, virtio_max_channels, virtio_expiration_time))
> +         return -1;
> +
> +      rc = pthread_create(&virtio_tid, NULL, virtio_run, NULL);
> +
> +      if (rc != 0) {
> +         vu_log(VHOSTMD_ERR, "Failed to start virtio thread '%s'\n",
> +                strerror(rc));
> +         return -1;
> +      }
> +   }
>      
>      while (!down) {
>         time_t run_time,
> @@ -948,6 +999,12 @@ static int vhostmd_run(int diskfd)
>         vu_buffer_erase(buf);
>      }
>      vu_buffer_delete(buf);
> +
> +   if (transports & VIRTIO) {
> +      virtio_stop();
> +      pthread_join(virtio_tid, NULL);
> +   }
> +
>      return 0;
>   }
>   
> 




More information about the virt-tools-list mailing list