[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Libguestfs] [PATCH v8 4/4] v2v: ova: don't extract files from OVA if it's not needed



On Sat, Feb 04, 2017 at 03:10:46PM +0100, Tomáš Golembiovský wrote:
> We don't have to always extract all files from the OVA archive. The OVA,
> as defined in the standard, is plain tar. We can work directly over the
> tar archive if we use correct 'offset' and 'size' options when defining
> the backing file for QEMU. This puts much lower requirement on available
> disk space.
> 
> Since the virt-v2v behaviour for OVA input now depends on QEMU version
> available this affects some of the tests. Expected result of the
> affected also has to depend on the QEMU used thus such tests will have
> two *.expected files.
> 
> Signed-off-by: Tomáš Golembiovský <tgolembi redhat com>

This commit (I think) breaks virt-v2v when run with the libvirt
backend using an OVA input file.  The error from v2v/test-v2v-i-ova.sh
is:

  Original error from libvirt: internal error: missing parser implementation 
  for JSON backing volume driver 'raw' [code=1 int1=-1]

which I think is caused because libvirt doesn't understand a backing
file with the format 'json:{"driver":"raw",...}'.  Full log is
attached.  This is similar but not exactly the same as this bug:

  https://bugzilla.redhat.com/show_bug.cgi?id=1134878

I have added a workaround in the test for now, but please look at
whether this can be fixed more permanently.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_tmpdir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_tmpdir = 0
libguestfs: trace: set_cachedir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_cachedir = 0
libguestfs: trace: set_path "/home/rjones/d/libguestfs/appliance"
libguestfs: trace: set_path = 0
libguestfs: create: flags = 0, handle = 0x55c0a01e50a0, program = guestfish
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "libvirt"
libguestfs: trace: shutdown
libguestfs: trace: shutdown = 0
libguestfs: trace: close
libguestfs: closing guestfs handle 0x55c0a01e50a0 (state 0)
~/d/libguestfs/v2v ~/d/libguestfs/v2v
~/d/libguestfs/v2v
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_tmpdir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_tmpdir = 0
libguestfs: trace: set_cachedir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_cachedir = 0
libguestfs: trace: set_path "/home/rjones/d/libguestfs/appliance"
libguestfs: trace: set_path = 0
libguestfs: create: flags = 0, handle = 0x10c4c60, program = virt-v2v
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_tmpdir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_tmpdir = 0
libguestfs: trace: set_cachedir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_cachedir = 0
libguestfs: trace: set_path "/home/rjones/d/libguestfs/appliance"
libguestfs: trace: set_path = 0
libguestfs: create: flags = 0, handle = 0x10c50b0, program = virt-v2v
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_tmpdir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_tmpdir = 0
libguestfs: trace: set_cachedir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_cachedir = 0
libguestfs: trace: set_path "/home/rjones/d/libguestfs/appliance"
libguestfs: trace: set_path = 0
libguestfs: create: flags = 0, handle = 0x10c5660, program = virt-v2v
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/home/rjones/d/libguestfs/tmp"
[   0.0] Opening the source -i ova test-v2v-i-ova.d/test-ova.ova
[   0.1] Creating an overlay to protect the source from being modified
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_tmpdir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_tmpdir = 0
libguestfs: trace: set_cachedir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_cachedir = 0
libguestfs: trace: set_path "/home/rjones/d/libguestfs/appliance"
libguestfs: trace: set_path = 0
libguestfs: create: flags = 0, handle = 0x11406d0, program = virt-v2v
libguestfs: trace: disk_has_backing_file "/home/rjones/d/libguestfs/tmp/v2vovlb76359.qcow2"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ info
libguestfs: command: run: \ --output json
libguestfs: command: run: \ /dev/fd/3
libguestfs: parse_json: qemu-img info JSON output:\n{\n    "backing-filename-format": "vmdk",\n    "virtual-size": 536870912,\n    "filename": "/dev/fd/3",\n    "cluster-size": 65536,\n    "format": "qcow2",\n    "actual-size": 200704,\n    "format-specific": {\n        "type": "qcow2",\n        "data": {\n            "compat": "1.1",\n            "lazy-refcounts": false,\n            "refcount-bits": 16,\n            "corrupt": false\n        }\n    },\n    "full-backing-filename": "json:{ \"file\": { \"driver\": \"raw\", \"offset\": 10752, \"size\": 4063232, \"file\": { \"filename\": \"test-v2v-i-ova.d/test-ova.ova\" } } }",\n    "backing-filename": "json:{ \"file\": { \"driver\": \"raw\", \"offset\": 10752, \"size\": 4063232, \"file\": { \"filename\": \"test-v2v-i-ova.d/test-ova.ova\" } } }",\n    "dirty-flag": false\n}\n\n
libguestfs: trace: disk_has_backing_file = 1
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_tmpdir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_tmpdir = 0
libguestfs: trace: set_cachedir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_cachedir = 0
libguestfs: trace: set_path "/home/rjones/d/libguestfs/appliance"
libguestfs: trace: set_path = 0
libguestfs: create: flags = 0, handle = 0x113fde0, program = virt-v2v
libguestfs: trace: disk_virtual_size "/home/rjones/d/libguestfs/tmp/v2vovlb76359.qcow2"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ info
libguestfs: command: run: \ --output json
libguestfs: command: run: \ /dev/fd/3
libguestfs: parse_json: qemu-img info JSON output:\n{\n    "backing-filename-format": "vmdk",\n    "virtual-size": 536870912,\n    "filename": "/dev/fd/3",\n    "cluster-size": 65536,\n    "format": "qcow2",\n    "actual-size": 200704,\n    "format-specific": {\n        "type": "qcow2",\n        "data": {\n            "compat": "1.1",\n            "lazy-refcounts": false,\n            "refcount-bits": 16,\n            "corrupt": false\n        }\n    },\n    "full-backing-filename": "json:{ \"file\": { \"driver\": \"raw\", \"offset\": 10752, \"size\": 4063232, \"file\": { \"filename\": \"test-v2v-i-ova.d/test-ova.ova\" } } }",\n    "backing-filename": "json:{ \"file\": { \"driver\": \"raw\", \"offset\": 10752, \"size\": 4063232, \"file\": { \"filename\": \"test-v2v-i-ova.d/test-ova.ova\" } } }",\n    "dirty-flag": false\n}\n\n
libguestfs: trace: disk_virtual_size = 536870912
[   0.1] Initializing the target -o local -os test-v2v-i-ova.d
[   0.1] Opening the overlay
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_tmpdir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_tmpdir = 0
libguestfs: trace: set_cachedir "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: set_cachedir = 0
libguestfs: trace: set_path "/home/rjones/d/libguestfs/appliance"
libguestfs: trace: set_path = 0
libguestfs: create: flags = 0, handle = 0x1140130, program = virt-v2v
libguestfs: trace: set_identifier "v2v"
libguestfs: trace: v2v: set_identifier = 0
libguestfs: trace: v2v: get_memsize
libguestfs: trace: v2v: get_memsize = 500
libguestfs: trace: v2v: set_memsize 2000
libguestfs: trace: v2v: set_memsize = 0
libguestfs: trace: v2v: set_network true
libguestfs: trace: v2v: set_network = 0
libguestfs: trace: v2v: add_drive "/home/rjones/d/libguestfs/tmp/v2vovlb76359.qcow2" "format:qcow2" "cachemode:unsafe" "discard:besteffort" "copyonread:true"
libguestfs: trace: v2v: add_drive = 0
libguestfs: trace: v2v: launch
libguestfs: trace: v2v: get_tmpdir
libguestfs: trace: v2v: get_tmpdir = "/home/rjones/d/libguestfs/tmp"
libguestfs: trace: v2v: version
libguestfs: trace: v2v: version = <struct guestfs_version = major: 1, minor: 35, release: 21, extra: , >
libguestfs: trace: v2v: get_backend
libguestfs: trace: v2v: get_backend = "libvirt"
libguestfs: launch: program=virt-v2v
libguestfs: launch: identifier=v2v
libguestfs: launch: version=1.35.21
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=libvirt
libguestfs: launch: tmpdir=/home/rjones/d/libguestfs/tmp/libguestfsPT4q2Y
libguestfs: launch: umask=0002
libguestfs: launch: euid=1000
libguestfs: libvirt version = 3000000 (3.0.0)
libguestfs: guest random name = guestfs-rkoxuu2n9eeycio5
libguestfs: connect to libvirt
libguestfs: opening libvirt handle: URI = qemu:///session, auth = default+wrapper, flags = 0
libguestfs: successfully opened libvirt handle: conn = 0x1177320
libguestfs: qemu version (reported by libvirt) = 2008000 (2.8.0)
libguestfs: get libvirt capabilities
libguestfs: parsing capabilities XML
libguestfs: trace: v2v: get_backend_setting "force_tcg"
libguestfs: trace: v2v: get_backend_setting = NULL (error)
libguestfs: trace: v2v: get_backend_setting "internal_libvirt_label"
libguestfs: trace: v2v: get_backend_setting = NULL (error)
libguestfs: trace: v2v: get_backend_setting "internal_libvirt_imagelabel"
libguestfs: trace: v2v: get_backend_setting = NULL (error)
libguestfs: trace: v2v: get_backend_setting "internal_libvirt_norelabel_disks"
libguestfs: trace: v2v: get_backend_setting = NULL (error)
libguestfs: trace: v2v: get_backend_setting "network_bridge"
libguestfs: trace: v2v: get_backend_setting = NULL (error)
libguestfs: build appliance
libguestfs: trace: v2v: get_cachedir
libguestfs: trace: v2v: get_cachedir = "/home/rjones/d/libguestfs/tmp"
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /home/rjones/d/libguestfs/tmp/.guestfs-1000/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /home/rjones/d/libguestfs/appliance/supermin.d
libguestfs: command: run: \ -o /home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d
supermin: version: 5.1.17
supermin: rpm: detected RPM version 4.13
supermin: package handler: fedora/rpm
supermin: acquiring lock on /home/rjones/d/libguestfs/tmp/.guestfs-1000/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: trace: v2v: disk_create "/home/rjones/d/libguestfs/tmp/libguestfsPT4q2Y/overlay1" "qcow2" -1 "backingfile:/home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/root" "backingformat:raw"
libguestfs: command: run: qemu-img
libguestfs: command: run: \ create
libguestfs: command: run: \ -f qcow2
libguestfs: command: run: \ -o backing_file=/home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/root,backing_fmt=raw
libguestfs: command: run: \ /home/rjones/d/libguestfs/tmp/libguestfsPT4q2Y/overlay1
Formatting '/home/rjones/d/libguestfs/tmp/libguestfsPT4q2Y/overlay1', fmt=qcow2 size=4294967296 backing_file=/home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/root backing_fmt=raw encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
libguestfs: trace: v2v: disk_create = 0
libguestfs: trace: v2v: get_sockdir
libguestfs: trace: v2v: get_sockdir = "/run/user/1000"
libguestfs: create libvirt XML
libguestfs: trace: v2v: get_cachedir
libguestfs: trace: v2v: get_cachedir = "/home/rjones/d/libguestfs/tmp"
libguestfs: libvirt XML:\n<?xml version="1.0"?>\n<domain type="kvm" xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0";>\n  <name>guestfs-rkoxuu2n9eeycio5</name>\n  <memory unit="MiB">2000</memory>\n  <currentMemory unit="MiB">2000</currentMemory>\n  <cpu mode="host-passthrough">\n    <model fallback="allow"/>\n  </cpu>\n  <vcpu>1</vcpu>\n  <clock offset="utc">\n    <timer name="rtc" tickpolicy="catchup"/>\n    <timer name="pit" tickpolicy="delay"/>\n    <timer name="hpet" present="no"/>\n  </clock>\n  <os>\n    <type>hvm</type>\n    <kernel>/home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/kernel</kernel>\n    <initrd>/home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d/initrd</initrd>\n    <cmdline>panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 guestfs_network=1 TERM=xterm-256color guestfs_identifier=v2v</cmdline>\n    <bios useserial="yes"/>\n  </os>\n  <on_reboot>destroy</on_reboot>\n  <devices>\n    <rng model="virtio">\n      <backend model="random">/dev/urandom</backend>\n    </rng>\n    <controller type="scsi" index="0" model="virtio-scsi"/>\n    <disk device="disk" type="file">\n      <source file="/home/rjones/d/libguestfs/tmp/v2vovlb76359.qcow2"/>\n      <target dev="sda" bus="scsi"/>\n      <driver name="qemu" type="qcow2" cache="unsafe" discard="unmap"/>\n      <address type="drive" controller="0" bus="0" target="0" unit="0"/>\n    </disk>\n    <disk type="file" device="disk">\n      <source file="/home/rjones/d/libguestfs/tmp/libguestfsPT4q2Y/overlay1"/>\n      <target dev="sdb" bus="scsi"/>\n      <driver name="qemu" type="qcow2" cache="unsafe"/>\n      <address type="drive" controller="0" bus="0" target="1" unit="0"/>\n      <shareable/>\n    </disk>\n    <serial type="unix">\n      <source mode="connect" path="/run/user/1000/libguestfsnN9AkE/console.sock"/>\n      <target port="0"/>\n    </serial>\n    <channel type="unix">\n      <source mode="connect" path="/run/user/1000/libguestfsnN9AkE/guestfsd.sock"/>\n      <target type="virtio" name="org.libguestfs.channel.0"/>\n    </channel>\n    <interface type="bridge">\n      <source bridge="virbr0"/>\n      <model type="virtio"/>\n    </interface>\n    <controller type="usb" model="none"/>\n    <memballoon model="none"/>\n  </devices>\n  <qemu:commandline>\n    <qemu:env name="TMPDIR" value="/home/rjones/d/libguestfs/tmp"/>\n  </qemu:commandline>\n</domain>\n
libguestfs: trace: v2v: get_cachedir
libguestfs: trace: v2v: get_cachedir = "/home/rjones/d/libguestfs/tmp"
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -R
libguestfs: command: run: \ -Z /home/rjones/d/libguestfs/tmp/.guestfs-1000
libguestfs: /home/rjones/d/libguestfs/tmp/.guestfs-1000:
libguestfs: total 92
libguestfs: drwxr-xr-x. 3 rjones rjones unconfined_u:object_r:user_tmp_t:s0  4096 Feb  7 21:02 .
libguestfs: drwxrwxr-x. 6 rjones rjones system_u:object_r:tmp_t:s0          45056 Feb  7 21:02 ..
libguestfs: drwxr-xr-x. 2 rjones rjones unconfined_u:object_r:user_tmp_t:s0  4096 Feb  7 20:58 appliance.d
libguestfs: -rw-r--r--. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0     0 Feb  7 17:41 lock
libguestfs: -rw-rw-r--. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0 10027 Feb  7 18:26 qemu.devices
libguestfs: -rw-rw-r--. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0 23588 Feb  7 18:26 qemu.help
libguestfs: -rw-rw-r--. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0    35 Feb  7 18:26 qemu.stat
libguestfs: 
libguestfs: /home/rjones/d/libguestfs/tmp/.guestfs-1000/appliance.d:
libguestfs: total 422532
libguestfs: drwxr-xr-x. 2 rjones rjones unconfined_u:object_r:user_tmp_t:s0       4096 Feb  7 20:58 .
libguestfs: drwxr-xr-x. 3 rjones rjones unconfined_u:object_r:user_tmp_t:s0       4096 Feb  7 21:02 ..
libguestfs: -rw-r--r--. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0     521728 Feb  7 21:02 initrd
libguestfs: -rwxr-xr-x. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0    7172184 Feb  7 21:02 kernel
libguestfs: -rw-r--r--. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0 4294967296 Feb  7 21:02 root
libguestfs: command: run: ls
libguestfs: command: run: \ -a
libguestfs: command: run: \ -l
libguestfs: command: run: \ -Z /run/user/1000/libguestfsnN9AkE
libguestfs: total 0
libguestfs: drwx------. 2 rjones rjones unconfined_u:object_r:user_tmp_t:s0  80 Feb  7 21:02 .
libguestfs: drwx------. 7 rjones rjones system_u:object_r:user_tmp_t:s0     160 Feb  7 21:02 ..
libguestfs: srwxrwxr-x. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0   0 Feb  7 21:02 console.sock
libguestfs: srwxrwxr-x. 1 rjones rjones unconfined_u:object_r:user_tmp_t:s0   0 Feb  7 21:02 guestfsd.sock
libguestfs: launch libvirt guest
libguestfs: trace: v2v: launch = -1 (error)
virt-v2v: error: libguestfs error: could not create appliance through 
libvirt.

Try running qemu directly without libvirt using this environment variable:
export LIBGUESTFS_BACKEND=direct

Original error from libvirt: internal error: missing parser implementation 
for JSON backing volume driver 'raw' [code=1 int1=-1]

If reporting bugs, run virt-v2v with debugging enabled and include the 
complete output:

  virt-v2v -v -x [...]
libguestfs: trace: close
libguestfs: closing guestfs handle 0x10c4c60 (state 0)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x10c50b0 (state 0)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x10c5660 (state 0)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x11406d0 (state 0)
libguestfs: trace: close
libguestfs: closing guestfs handle 0x113fde0 (state 0)
libguestfs: trace: v2v: close
libguestfs: closing guestfs handle 0x1140130 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /home/rjones/d/libguestfs/tmp/libguestfsPT4q2Y
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /run/user/1000/libguestfsnN9AkE
FAIL test-v2v-i-ova.sh (exit status: 1)

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]