[libvirt] RFC: solving problem with qemu domain save on root-squashing NFS

[Laine Stump]

(I had sent this message last week as a response to a message in a patch 
thread, but thought today that possibly some mail readers would bury it, 
so I'm sending it again disconnected from any existing thread. I can 
send the patches I mention below to anyone interested in playing around 
with them, but they're not useful since they still fail at the final 
step...)

On 01/24/2010 11:57 AM, Dan Kenigsberg wrote:
> On Wed, Jan 20, 2010 at 03:14:57PM +0000, Daniel P. Berrange wrote:
>> This patch series does some work on te security drivers, and the QEMU 
>> code
>> for managing DAC permissions on files.
>>
>> The core goal is to turn the QEMU driver DAC file management code into a
>> security driver. Instead of QEMU calling into the SELinux/AppArmour 
>> drivers
>> directly, a stacked driver module is introduced. This delegates all 
>> operations
>> to first the QEMU DAC driver, and then the main SELinux/AppArmour 
>> driver.
>> The end result is that all the permissions management code is removed 
>> from
>> the QEMU driver, and we're left with just simple security driver calls.
>>
>> In the process of this a number of flaws in the current hotplug  code 
>> were
>> found, and code was generally tidied up with a view to making it 
>> easier to
>> manage.
>>
>> Finally, we add the ability to turn off the QEMU  DAC file managment 
>> code,
>> and also deal gracefully with failures to change ownership (eg on NFS 
>> with
>> root squash, or readonly FS).
> Thanks for this series. However, it seems that we still have a problem
> when trying to save domain to a root-squashing nfs export.
> When using qemu directly, as a user with write permissions to that
> export, there is no problem. When using libvirt, libvirt tries to write
> its own state to the target file. I would not want to pre-create the
> target file as world redable.
>
> How about performing open(path, O_CREAT|O_TRUNC|O_WRONLY,
> S_IRUSR|S_IWUSR)) with the euid of the qemu process?

I came up with a patch that creates the domain state file by first 
forking, followed by the child process doing setuid(qemu) and all the 
write-to-the-file stuff that creates the file (similar to the recent 
virFileCreate patch). This works all the way up through qemu's appending 
its state onto the file.

The trouble comes when qemudDomainSave calls 
securityDriver->domainRestoreSavedStateLabel() (in this case, 
qemuSecurityStackedRestoreSavedStateLabel()). That function first calls 
the primary security driver's qemuSecurityDACRestoreSavedStateLabel(), 
which will fail because it attempts to chown the file to 0,0 (to prevent 
other qemu instances from messing with it). That can be disabled by 
setting "dynamic_ownership=0" in qemu.conf (needed in several other 
places when stuff is stored on root-squashing NFS anyway :-/). Then the 
stacked security driver calls the secondary security driver's 
SELinuxRestoreSavedStateLabel(), which also fails because of an SELinux 
function (matchpathcon) returning failure. We could ignore this failure, 
but that would create an unnecessary security hole for all those 
installations that *aren't* storing their domains on root-squashing NFS.

So I *think* what I'm looking for a good way to avoid the SELinux stuff 
when the destination file is located on a root-squashing NFS share, but 
not ignore it otherwise (I believe the consensus is that we *don't* want 
to require another configuration knob if at all possible, since that 
would not only lead to extra complexity of config, but also to 
uninformed users turning it off when not necessary). Does anyone have 
any good ideas for doing this (or for a different approach)? (I suppose 
I could always attempt the entire operation assuming not, then redo with 
fork/setuid if the first attempt failed, but that seems less than elegant).