[Libguestfs] Performance issue: "udevadm settle" is called three times for a single call of guestfs_download_offset() (#67)

[Richard W.M. Jones]

On Mon, May 17, 2021 at 10:21:33PM -0700, Motohiro Kawahito wrote:
> I'm planning to use guestfs_download_offset() to read a part of
> /dev/sda. I made a micro-benchmark for it.

What is the host file?  Raw?  qcow2?

> I found that a single 512-bytes read access takes 60msec. It is very slow.

The whole architecture is not designed for performance in this sort of
case.  Essentially it is single threaded and making a round trip on
every 512 byte request.  See:

  https://libguestfs.org/guestfs-internals.1.html#architecture

> I investigated a reason of it. I found that "udevadm settle -E /dev/sda" is
> invoked 3 times and that these three calls take 57msec.
> 
> The command udevadm is called from the beginning of is_device_parameter(), as
> shown below.
> https://github.com/libguestfs/libguestfs/blob/047cf7dcd26e649d45e7e21a3b679bad0bb6c312/daemon/utils.c#L120
> 
> I have the following two questions:
> 
>   • Can we defer calling udev_settle_file (device) after next two IFs? (That
>     is, after Line 128) I tried it, and the performance was improved to 3msec.

Yes, that should be fine, although I don't think it will make your
case faster.

>   • Can we cache the result of is_device_parameter() and then use it for the
>     other two calls?

Possibly.  Do you know where it is being called from 3 times?

Stepping back, I would say that this is the wrong way to go about this
(unless you just want to benchmark libguestfs itself, which will only
tell you that it's slow).  If the host file is raw, you can just read
it.  If the host file is qcow2, try something like this instead, it
will be far faster:

  $ nbdsh \
      -c 'h.connect_systemd_socket_activation(["qemu-nbd", "-r", "-f", "qcow2", "test.qcow2"])' \
      -c 'print ("%r" % h.pread(512,0))'

You could also write it in C:
https://rwmj.wordpress.com/2019/10/03/how-to-edit-a-qcow2-file-from-c/

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html