[libvirt-users] using RBD with libvirt 0.9.13
Jeff Strunk
jstrunk at math.utexas.edu
Mon Aug 6 20:45:37 UTC 2012
Thank you it works now.
The issue was the lack of the auth element.
qemu has access to /etc/ceph/ceph.conf. Specifying the host elements in the source element caused a crash:
error: Failed to start domain test0
error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/3
server name not found: thinkmate3:6789;thinkmate4:6789 (No such file or directory)
unable to parse addrs in 'thinkmate3:6789;thinkmate4:6789'
mon/MonClient.cc: In function 'void MonClient::_pick_new_mon()' thread 7fd1de5d67c0 time 2012-08-06 15:40:38.191920
mon/MonClient.cc: 424: FAILED assert(monmap.size() > 0)
ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030)
1: (MonClient::_pick_new_mon()+0x40a) [0x7fd1dd6f4e8a]
2: (MonClient::_reopen_session()+0x187) [0x7fd1dd6f8c47]
3: (MonClient::authenticate(double)+0x1aa) [0x7fd1dd6f9fea]
4: (librados::RadosClient::connect()+0xb6c) [0x7fd1dd63d6ec]
5: (()+0x94393) [0x7fd1de698393]
6: (()+0x801e5) [0x7fd1de6841e5]
7: (()+0x80366) [0x7fd1de684366]
8: (()+0x802d9) [0x7fd1de6842d9]
9: (()+0x80ee7) [0x7fd1de684ee7]
10: (()+0xa3b03) [0x7fd1de6a7b03]
11: (()+0xfec4b) [0x7fd1de702c4b]
12: (()+0x11fbc2
On Monday, August 06, 2012 01:33:34 PM Josh Durgin wrote:
> On 08/06/2012 12:12 PM, Jeff Strunk wrote:
> > I'm having some trouble creating KVM domains with RBD block devices using
> > virsh. I've managed to get virsh to define the domain, but it gives an
> > error when trying to start the domain:
> > error: Failed to start domain test0
> > error: internal error process exited while connecting to monitor: char
> > device redirected to /dev/pts/3 kvm: -drive
> > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: error
> > connecting
> This is the relevant error. If you don't have an /etc/ceph/ceph.conf
> file, or it (or the keyring file possibly) is unreadable by qemu due to
> apparmor/selinux etc, you'll need to specify all the information needed
> to connect in the drive xml. This includes the monitor host and the
> libvirt secret uuid, i.e.:
>
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw'/>
> <source protocol='rbd' name='rbd/test0'>
> <host name='thinkmate3' port='6789'/>
> <host name='thinkmate4' port='6789'/>
> </source>
> <auth username='admin'>
> <secret type='ceph' uuid='f0836fcb-9c05-5ccf-653a-eb1d0308e247'/>
> </auth>
> <target dev='vda' bus='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </disk>
>
> Josh
>
> > kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw:
> > could not open disk image rbd:rbd/test0: No such file or directory>
> >>From libvirt's QEMU log:
> > 2012-08-06 18:36:57.788+0000: starting up
> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
> > QEMU_AUDIO_DRV=none /usr/bin/kvm -name test0 -S -M pc-1.0 -enable-kvm -m
> > 1024 -smp 1,sockets=1,cores=1,threads=1 -uuid
> > 3e727624-f336-d674-4e0c-234641fc9987 -nodefconfig -nodefaults -chardev
> > socket,id=charmonitor,path=/var/lib/libvirt/qemu/test0.monitor,server,now
> > ait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
> > -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
> > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw -device
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=vi
> > rtio-disk0,bootindex=1 -drive
> > if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev
> > tap,fd=19,id=hostnet0 -device
> > virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:8c:e4:92,bus=pci.0,ad
> > dr=0x3 -chardev pty,id=charserial0 -device
> > isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -vga cirrus -!>
> > device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
> >
> > Domain id=5 is tainted: high-privileges
> > char device redirected to /dev/pts/2
> > kvm: -drive file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw:
> > error connecting kvm: -drive
> > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw: could not
> > open disk image rbd:rbd/test0: No such file or directory 2012-08-06
> > 18:36:58.353+0000: shutting down
> >
> > I can, however, successfully run a KVM virtual machine with an RBD block
> > device with the following command:
> > kvm -drive file=rbd:rbd/test1,id=drive-virtio-disk0,format=raw -cdrom
> > /var/lib/libvirt/images/ubuntu-12.04-server-amd64.iso
> >
> > kvm will boot with the "-drive
> > file=rbd:rbd/test0,if=none,id=drive-virtio-disk0,format=raw" options, but
> > the disk is not detected unless "if=none" is removed.
> >
> > Has anyone else figured this out?
> >
> > Thank you,
> > Jeff
> >
> > Here is my rbd pool definition:
> > ######### BEGIN
> > <pool type="rbd">
> > <name>rbd</name>
> > <source>
> > <name>rbd</name>
> > <host name='thinkmate3' port='6789'/>
> > <host name='thinkmate4' port='6789'/>
> > <auth username='admin' type='ceph'>
> > <secret uuid='f0836fcb-9c05-5ccf-653a-eb1d0308e247'/>
> > </auth>
> > </source>
> > </pool>
> > ######### END
> >
> > My rbd volumes:
> > # virsh vol-list rbd
> > Name Path
> > -----------------------------------------
> > test0 rbd/test0
> > test1 rbd/test1
> > test2 rbd/test2
> >
> > Here is my domain definition:
> > ######### BEGIN
> > <domain type='kvm'>
> >
> > <name>test0</name>
> > <uuid>3e727624-f336-d674-4e0c-234641fc9987</uuid>
> > <memory unit='KiB'>1048576</memory>
> > <currentMemory unit='KiB'>1048576</currentMemory>
> > <vcpu placement='static'>1</vcpu>
> > <os>
> >
> > <type arch='x86_64' machine='pc-1.0'>hvm</type>
> > <boot dev='hd'/>
> >
> > </os>
> > <features>
> >
> > <acpi/>
> > <apic/>
> > <pae/>
> >
> > </features>
> > <clock offset='utc'/>
> > <on_poweroff>destroy</on_poweroff>
> > <on_reboot>restart</on_reboot>
> > <on_crash>restart</on_crash>
> > <devices>
> >
> > <emulator>/usr/bin/kvm</emulator>
> > <disk type='network' device='disk'>
> >
> > <driver name='qemu' type='raw'/>
> > <source protocol='rbd' name='rbd/test0'/>
> > <target dev='vda' bus='virtio'/>
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> > function='0x0'/>
> >
> > </disk>
> > <disk type='block' device='cdrom'>
> >
> > <driver name='qemu' type='raw'/>
> > <target dev='hdc' bus='ide'/>
> > <readonly/>
> > <address type='drive' controller='0' bus='1' target='0' unit='0'/>
> >
> > </disk>
> > <controller type='ide' index='0'>
> >
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> > function='0x1'/>
> >
> > </controller>
> > <controller type='usb' index='0'>
> >
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> > function='0x2'/>
> >
> > </controller>
> > <interface type='network'>
> >
> > <mac address='52:54:00:8c:e4:92'/>
> > <source network='default'/>
> > <model type='virtio'/>
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> > function='0x0'/>
> >
> > </interface>
> > <serial type='pty'>
> >
> > <target port='0'/>
> >
> > </serial>
> > <console type='pty'>
> >
> > <target type='serial' port='0'/>
> >
> > </console>
> > <input type='mouse' bus='ps2'/>
> > <graphics type='vnc' port='-1' autoport='yes'/>
> > <video>
> >
> > <model type='cirrus' vram='9216' heads='1'/>
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> > function='0x0'/>
> >
> > </video>
> > <memballoon model='virtio'>
> >
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> > function='0x0'/>
> >
> > </memballoon>
> >
> > </devices>
> >
> > </domain>
> > ######## END
More information about the libvirt-users
mailing list