[Libguestfs] qemu copyonread vs trimming?

[Richard W.M. Jones]

Hi Eric,

I wonder if you'd know what the difference is here:

$ ./nbdkit -U - --filter=cow --filter=xz \
  curl https://download.fedoraproject.org/pub/fedora/linux/releases/34/Cloud/x86_64/images/Fedora-Cloud-Base-34-1.2.x86_64.raw.xz \
  --run '
    guestfish add "" format:raw protocol:nbd server:unix:$unixsocket cachemode:unsafe discard:enable copyonread:true : run : mount /dev/sda1 / : fstrim / &&
    nbdinfo --map $uri
  '
         0  5368709120    0  data

(trimming didn't happen)

Changing copyonread:true to copyonread:false and keeping everything
else identical, the output changes to:

         0    38010880    0  data
  38010880      917504    3  hole,zero
  38928384     6881280    0  data
  45809664    89456640    3  hole,zero
 135266304     8781824    0  data
 [...]

(trimming happened)

There's a more detailed analysis below, but at first glance it appears
as if the copy-on-read filter in qemu doesn't pass trim requests.  In
the nbdkit verbose output in the first case I don't see trim at all.
In the second (working) case, I see trim requests.

What's weird is that this case comes from a problem with modular
virt-v2v, yet old virt-v2v worked in the same way, except we're using
the qemu nbd client here but old virt-v2v would have used a qcow2 file.

This is with qemu-6.0.0-10.fc35.x86_64.  I checked the qemu sources
and as far as I can tell discards (cor_co_pdiscard?) get passed
through.

======================================================================

MORE DETAIL: What's happening is that guestfish uses libvirt to set up
these disks.

With copyonread:true:

    <disk device="disk" type="network">
      <source protocol="nbd">
        <host transport="unix" socket="/tmp/nbdkit4IH9fV/socket"/>
      </source>
      <target dev="sda" bus="scsi"/>
      <driver name="qemu" type="raw" cache="unsafe" discard="unmap" copy_on_read="on"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <disk type="file" device="disk">

with copyonread:false:

    <disk device="disk" type="network">
      <source protocol="nbd">
        <host transport="unix" socket="/tmp/nbdkitxmjFrS/socket"/>
      </source>
      <target dev="sda" bus="scsi"/>
      <driver name="qemu" type="raw" cache="unsafe" discard="unmap"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>

These translate to the following qemu command lines (copyonread:true):

  -blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/nbdkit4IH9fV/socket"},"node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":true},"auto-read-only":true,"discard":"unmap"}' \
  -blockdev '{"node-name":"libvirt-2-format","read-only":false,"discard":"unmap","cache":{"direct":false,"no-flush":true},"driver":"raw","file":"libvirt-2-storage"}' \
  -blockdev '{"driver":"copy-on-read","node-name":"libvirt-CoR-sda","file":"libvirt-2-format"}' \
  -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-CoR-sda,id=scsi0-0-0-0,bootindex=1,write-cache=on \

or (copyonread:false):

  -blockdev '{"driver":"nbd","server":{"type":"unix","path":"/tmp/nbdkitxmjFrS/socket"},"node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":true},"auto-read-only":true,"discard":"unmap"}' \
  -blockdev '{"node-name":"libvirt-2-format","read-only":false,"discard":"unmap","cache":{"direct":false,"no-flush":true},"driver":"raw","file":"libvirt-2-storage"}' \
  -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-2-format,id=scsi0-0-0-0,bootindex=1,write-cache=on \


These seem very similar to me, except there's an extra copy-on-read
layer in the first (non-working) case.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top