[libvirt] [libvirt-users] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

Tue Apr 12 14:58:43 UTC 2016

Hey Martin,

Thanks very much.  Appreciate you jumping in on this thread.

You see, that's just it.  I've configured libvirt .conf files to run as 
oneadmin.oneadmin (non previlidged) for that NFS share and I can access 
all the files on that share as oneadmin without error, including the one 
you listed.  But libvirtd, by default, always starts as root.  So it's 
doing something as root, despite being configured to access the share as 
oneadmin.  As oneadmin I can access that file no problem.  Here's how I 
read the file off the node on which the NFS share is mounted on:

[oneadmin at mdskvm-p01 ~]$ ls -altri /var/lib/one//datastores/0/38/disk.1
34642274 -rw-r--r-- 1 oneadmin oneadmin 372736 Apr 5 00:20 
/var/lib/one//datastores/0/38/disk.1
[oneadmin at mdskvm-p01 ~]$ file /var/lib/one//datastores/0/38/disk.1
/var/lib/one//datastores/0/38/disk.1: # ISO 9660 CD-ROM filesystem data 
'CONTEXT'
[oneadmin at mdskvm-p01 ~]$ strings /var/lib/one//datastores/0/38/disk.1|head
CD001
LINUX CONTEXT
GENISOIMAGE ISO 9660/HFS FILESYSTEM CREATOR (C) 1993 E.YOUNGDALE (C) 
1997-2006 J.PEARSON/J.SCHILLING (C) 2006-2007 CDRKIT TEAM 2016040500205600
2016040500205600
0000000000000000
2016040500205600

CD001
2016040500205600
2016040500205600
[oneadmin at mdskvm-p01 ~]$

My NFS mount looks as follows ( I have to use root_squash for security 
reasons.  I'm sure it will work using no_root_squash but that option is 
not an option here.):

[root at mdskvm-p01 ~]# grep nfs /etc/fstab
# 192.168.0.70:/var/lib/one/    /var/lib/one/  nfs 
context=system_u:object_r:nfs_t:s0,soft,intr,rsize=8192,wsize=8192,noauto
192.168.0.70:/var/lib/one/      /var/lib/one/  nfs 
soft,intr,rsize=8192,wsize=8192,noauto
[root at mdskvm-p01 ~]#

[root at opennebula01 ~]# cat /etc/exports
/var/lib/one/ *(rw,sync,no_subtree_check,root_squash)
[root at opennebula01 ~]#

So I dug deeper and see that there is a possibility libvirtd is trying 
to access that NFS mount as root as some level because as root I also 
get a permission denied to the NFS share above.  Rightly so since I have 
root_squash that I need to keep.  But libvirtd should be able to access 
the file as oneadmin as I have above.  It's not and this is what I read 
on it:

https://www.redhat.com/archives/libvir-list/2014-May/msg00194.html

Comment is: "The current implementation works for local
storage only and returns the canonical path of the volume."

But it seems the logic is applied to NFS mounts. Perhaps it shouldn't
be?  Anyway to get around this problem?  This is CentOS 7 .

My post with OpenNebula is here from which this conversation originates: 
https://forum.opennebula.org/t/libvirtd-running-as-root-tries-to-access-oneadmin-nfs-mount-error-cant-canonicalize-path/2054/7

Cheers,
Tom K.
------------------------------------------------------------------------------------- 

Living on earth is expensive, but it includes a free trip around the sun.

On 4/12/2016 10:03 AM, Martin Kletzander wrote:
> On Mon, Apr 11, 2016 at 08:02:04PM -0400, TomK wrote:
>> Hey All,
>>
>> Wondering if anyone had any suggestions on this topic?
>>
>
> The only thing I can come up with is:
> '/var/lib/one//datastores/0/38/disk.1': Permission denied
>
> ... that don't have access to that file.  Could you elaborate on that?
>
> I think it's either:
>
> a) you are running the domain as root or
>
> b) we don't use the domain's uid/gid to canonicalize the path.
>
> But if read access is enough for canonicalizing that path, I think the
> problem is purely with permissions.
>
>> Cheers,
>> Tom K.
>> ------------------------------------------------------------------------------------- 
>>
>>
>> Living on earth is expensive, but it includes a free trip around the 
>> sun.
>>
>> On 4/9/2016 11:08 AM, TomK wrote:
>>> Adding in libvir-list.
>>>
>>> Cheers,
>>> Tom K.
>>> ------------------------------------------------------------------------------------- 
>>>
>>>
>>> Living on earth is expensive, but it includes a free trip around the 
>>> sun.
>>>
>>> On 4/7/2016 7:32 PM, TomK wrote:
>>>> Hey All,
>>>>
>>>> I've an issue where libvirtd tries to access an NFS mount but errors
>>>> out with: can't canonicalize path '/var/lib/one//datastores/0 .  The
>>>> unprevilidged user is able to read/write fine to the share.
>>>> root_squash is used and for security reasons no_root_squash cannot be
>>>> used.
>>>>
>>>> On the controller and node SELinux is disabled.
>>>>
>>>> [oneadmin at mdskvm-p01 ~]$ virsh -d 1 --connect qemu:///system create
>>>> /var/lib/one//datastores/0/38/deployment.0
>>>> create: file(optdata): /var/lib/one//datastores/0/38/deployment.0
>>>> error: Failed to create domain from
>>>> /var/lib/one//datastores/0/38/deployment.0
>>>> error: can't canonicalize path
>>>> '/var/lib/one//datastores/0/38/disk.1': Permission denied
>>>>
>>>> I added some debug flags to get more info and added -x to the deploy
>>>> script. Closest I get to more details is this:
>>>>
>>>> 2016-04-06 04:15:35.945+0000: 14072: debug :
>>>> virStorageFileBackendFileInit:1441 : initializing FS storage file
>>>> 0x7f6aa4009000 (file:/var/lib/one//datastores/0/38/disk.1)[9869:9869]
>>>> 2016-04-06 04:15:35.954+0000: 14072: error :
>>>> virStorageFileBackendFileGetUniqueIdentifier:1523 : can't
>>>> canonicalize path '/var/lib/one//datastores/0/38/disk.1':
>>>>
>>>> https://www.redhat.com/archives/libvir-list/2014-May/msg00194.html
>>>>
>>>> Comment is: "The current implementation works for local
>>>> storage only and returns the canonical path of the volume."
>>>>
>>>> But it seems the logic is applied to NFS mounts. Perhaps it shouldn't
>>>> be?  Anyway to get around this problem?  This is CentOS 7 .
>>>>
>>>> Cheers,
>>>> Tom K.
>>>> ------------------------------------------------------------------------------------- 
>>>>
>>>>
>>>> Living on earth is expensive, but it includes a free trip around the
>>>> sun.
>>>>
>>>> _______________________________________________
>>>> libvirt-users mailing list
>>>> libvirt-users at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/libvirt-users
>>>
>>> -- 
>>> libvir-list mailing list
>>> libvir-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/libvir-list
>>
>> -- 
>> libvir-list mailing list
>> libvir-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/libvir-list
>
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users at redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20160412/8aa37b28/attachment-0001.htm>