[libvirt-users] [ceph-users] Setting RBD cache parameters for libvirt+qemu

[John Nielsen]

On Jun 7, 2013, at 5:01 PM, Josh Durgin <josh.durgin at inktank.com> wrote:

> On 06/07/2013 02:41 PM, John Nielsen wrote:
>> I am running some qemu-kvm virtual machines via libvirt using Ceph RBD as the back-end storage. Today I was testing an update to libvirt-1.0.6 on one of my hosts and discovered that it includes this change:
>> 	[libvirt] [PATCH] Forbid use of ':' in RBD pool names
>> 		...People are known to be abusing the lack of escaping in current libvirt to pass arbitrary args to QEMU.
>> 
>> I am one of those people--I have been storing colon-separated arguments in my disk source 'name' attribute to control the RBD cache settings on a per-volume basis. Under libvirt-1.0.6 such VM's refuse to start:
>> error: Failed to start domain foo
>> error: unsupported configuration: ':' not allowed in RBD source volume name 'mypool/foo:rbd_cache_size=268435456:rbd_cache_max_dirty=134217728:rbd_cache_target_dirty=33554432:rbd_cache_max_dirty_age=5'
>> 
>> So my question is how do I control the cache settings now?
> 
> You can set the cache mode with the cache attribute on the <driver>
> element inside a <disk> [1]. This works for rbd with QEMU 1.2 or later.
> 
> QEMU will try to read /etc/ceph/ceph.conf by default as well, so you
> can put more specific settings in a [global] or [client.foo] section
> there. Note that if you enable writeback caching here, you need to
> include cache='writeback' in the xml as well, or qemu will not send
> flushes.

Thanks for the quick reply. Is it safe to assume that the "foo" in "client.foo" should match the "auth username" attribute from the XML?

If I want different cache settings for different VM's or volumes on the same host is there a way to specify that any more? Can I specify multiple profiles in ceph.conf or point different qemu instances to different ceph.conf files?

JN