[libvirt-users] virt-sparsify changing the apparent-size of files
Richard W.M. Jones
rjones at redhat.com
Thu Jun 23 07:52:35 UTC 2016
On Wed, Jun 22, 2016 at 05:56:09PM -0500, libvirt_users at skagitattic.com wrote:
> > Delete what you've done and start from the beginning. Describe
> > exactly how you created the guest. Use 'qemu-img info' to show the
> > format of the input file. Show precisely the virt-sparsify command
> > you are running. And use 'qemu-img info' on the output file too.
>
> Okay, here goes. Start off with an install
>
> root at testingbox:~ # virt-install --name testimage --memory 512 -l
> 'http://ftp.nl.debian.org/debian/dists/jessie/main/installer-amd64/'
> --disk size=50 --nographics -x "console=ttyS0"
>
> Starting install...
> Retrieving file
> MANIFEST...
> | 3.3 kB 00:00 ... Retrieving file
> linux...
> | 6.0 MB 00:03 ... Retrieving file
> initrd.gz...
> | 29 MB 00:09 ... Allocating
> 'testimage.qcow2'
> | 50 GB 00:00 Creating
> domain...
> | 0 B 00:00 Connected to domain testimage Escape character is
> [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000]
> Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup
> subsys cpuacct [ 0.000000] Linux version 3.16.0-4-amd64
> (debian-kernel at lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1)
> ) #1 SMP Debian 3.16.7-ckt25-2 (2016-04-08)
>
> .... Many more lines here and it turns interactive. Suffice it to
> say I did a basic minimal install following the prompts. Nothing
> special here.
>
> This created an image that is 2.5G with a 50G apparent size and
> identifies as qcow2 as shown below.
>
> # ls -slh testimage.qcow2
> 2.5G -rw-r--r-- 1 root root 51G Jun 22 20:22 testimage.qcow2
>
> # qemu-img info testimage.qcow2
> image: testimage.qcow2
> file format: qcow2
> virtual size: 50G (53687091200 bytes)
> disk size: 2.4G
> cluster_size: 65536
> Format specific information:
> compat: 1.1
> lazy refcounts: true
>
> Now we sparsify (/bigtmp is on a larger partition then / to give
> virt-sparsify enough room to work with so it does not complain).
>
> root at testingbox: 08:59 PM # virt-sparsify testimage.qcow2
> testimage1.qcow2 --tmp /bigtmp Input disk virtual size = 53687091200
> bytes (50.0G) Create overlay file in /bigtmp to protect source
> disk ... Examine source disk ...
> 100%
> ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧
> --:-- Fill free space in /dev/sda1 with zero ... 100%
> ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧
> 00:00 Clearing Linux swap on /dev/sda5 ... 100%
> ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧
> --:-- Copy to destination and make sparse ...
>
> Sparsify operation completed with no errors. Before deleting the
> old disk, carefully check that the target disk boots and works
> correctly.
> root at testingbox: 09:04 PM #
>
> This image reports still reports as qcow2 however the apparent and real
> size is the same.
>
> # ls -slh testimage1.qcow2
> 1.1G -rw-r--r-- 1 root root 1.1G Jun 22 21:04 testimage1.qcow2
>
> # qemu-img info testimage1.qcow2
> image: testimage1.qcow2
> file format: qcow2
> virtual size: 50G (53687091200 bytes)
> disk size: 1.1G
> cluster_size: 65536
> Format specific information:
> compat: 1.1
> lazy refcounts: false
virt-sparsify saved about 1.3 GB of disk space (2.4G -> 1.1G).
> If we try it again but specify raw its MUCH faster
>
> root at testingbox: 09:26 PM # virt-sparsify testimage.qcow2
> testimage2.qcow2 --tmp /bigtmp --format raw
This is incorrect usage. You've told virt-sparsify that the input
image is raw. But it's not raw, it's qcow2.
In any case, virt-sparsify opened the image as raw, couldn't make any
sense of it (it appears to virt-sparsify to be random data, not a disk
image), and so it cannot sparsify it properly. This is "quicker" only
because virt-sparsify didn't do any sparsification of filesystems,
because it couldn't see the filesystems.
> This time it takes up more space and reports real and apparent size
> differently. It still reports as qcow2 with qemu-img.
>
> # ls -slh testimage2.qcow2
> 1.7G -rw-r--r-- 1 root root 51G Jun 22 21:27 testimage2.qcow2
> # qemu-img info testimage2.qcow2
> image: testimage2.qcow2
> file format: qcow2
> virtual size: 50G (53687091200 bytes)
> disk size: 1.7G
Looks like virt-sparsify still found some clusters containing all
zeroes which it could trim, even though it didn't know how to read the
disk.
Anyway, the moral is, don't misinform virt-sparsify of the format
of your input image.
If what you really intended to do was to convert the format from qcow2
to raw, you should use '--convert raw' instead.
> The only things I see different from the finished files is that the raw
> one uses a bit more space, the apparent size is not set to the
> virtual size on the default sparsify one and the lazy refcounts is
> different.
>
> Is there something more detailed then "qemu-img info" that I could tell
> the difference with?
qemu-img info is the correct tool to use.
> > Raw has fewer features, especially no simple snapshotting or support
> > for backing files.
I wrote this thinking you were talking about actual raw format disks,
not about qcow2 but where you've misinformed virt-sparsify about the
format.
> Is simple snapshotting using qemu-img snapshot or something else? I
> tried making a snapshot with both test images and they both appeared to
> work.
http://wiki.qemu.org/Documentation/CreateSnapshot
> # qemu-img snapshot -c test testimage1.qcow2
> # qemu-img snapshot -c test testimage2.qcow2
> # qemu-img info testimage2.qcow2
> image: testimage2.qcow2
> file format: qcow2
> virtual size: 50G (53687091200 bytes)
> disk size: 1.7G
> cluster_size: 65536
> Snapshot list:
> ID TAG VM SIZE DATE VM
> CLOCK 1 test 0 2016-06-22 22:41:52
> 00:00:00.000 Format specific information:
> compat: 1.1
> lazy refcounts: true
This works because testimage2.qcow2 is still in qcow2 format.
Anyway, use the correct --format option.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
More information about the libvirt-users
mailing list