[libvirt-users] using RBD with libvirt 0.9.13

[Jeff Strunk]

I'm having some trouble creating KVM domains with RBD block devices using
virsh. I've managed to get virsh to define the domain, but it gives an error
when trying to start the domain:
error: Failed to start domain test0
error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/3
kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: error connecting
kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: could not open disk image rbd:rbd/test0: No such file or directory

>From libvirt's QEMU log:
2012-08-06 18:36:57.788+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -name test0 -S -M pc-1.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid 3e727624-f336-d674-4e0c-234641fc9987 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/test0.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=19,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:8c:e4:92,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
Domain id=5 is tainted: high-privileges
char device redirected to /dev/pts/2
kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: error connecting
kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: could not open disk image rbd:rbd/test0: No such file or directory
2012-08-06 18:36:58.353+0000: shutting down

I can, however, successfully run a KVM virtual machine with an RBD block
device with the following command:
kvm -drive file=rbd:rbd/test1,id=drive-virtio-disk0,format=raw -cdrom /var/lib/libvirt/images/ubuntu-12.04-server-amd64.iso

kvm will boot with the "-drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw"
options, but the disk is not detected unless "if=none" is removed.

Has anyone else figured this out?

Thank you,
Jeff

Here is my rbd pool definition:
######### BEGIN
<pool type="rbd">
<name>rbd</name>
<source>
<name>rbd</name>
<host name='thinkmate3' port='6789'/>
<host name='thinkmate4' port='6789'/>
<auth username='admin' type='ceph'>
<secret uuid='f0836fcb-9c05-5ccf-653a-eb1d0308e247'/>
</auth>
</source>
</pool>
######### END

My rbd volumes:
# virsh vol-list rbd
Name                 Path                                    
-----------------------------------------
test0                rbd/test0                               
test1                rbd/test1                               
test2                rbd/test2 

Here is my domain definition:
######### BEGIN
<domain type='kvm'>
  <name>test0</name>
  <uuid>3e727624-f336-d674-4e0c-234641fc9987</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc-1.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw'/>
      <source protocol='rbd' name='rbd/test0'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='block' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='1' target='0' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:8c:e4:92'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>
######## END