[Virtio-fs] restorcon/SELinux virtiofs question

Harry G. Coin hgcoin at gmail.com
Sun Nov 29 21:41:00 UTC 2020 

On 11/20/20 12:55 PM, Vivek Goyal wrote:
> On Fri, Nov 20, 2020 at 11:11:28AM -0600, Harry G. Coin wrote:
>> On 11/20/20 9:01 AM, Daniel Walsh wrote:
>>> On 11/19/20 14:44, Vivek Goyal wrote:
>>>> On Thu, Nov 19, 2020 at 01:44:36PM -0500, Vivek Goyal wrote:
>>>>> On Thu, Nov 19, 2020 at 01:38:41PM -0500, Vivek Goyal wrote:
>>>>>> On Thu, Nov 19, 2020 at 12:27:20PM -0600, Harry G. Coin wrote:
>>>>>>> On 11/19/20 12:16 PM, Vivek Goyal wrote:
>>>>>>>> On Thu, Nov 19, 2020 at 10:52:51AM -0600, Harry G. Coin wrote:
>>>>>>>>> Hello virtiofs team.  I need clarification about a 'restorecon'
>>>>>>>>> selinux
>>>>>>>>> guest giving an 'operation not supported' response.
>>>>>>>>>
>>>>>>>>> If the host fs is btrfs (with xattr enabled in virtiofsd) but not
>>>>>>>>> running SELinux,
>>>>>>>> I suspect that on host setxattr(security.selinux) is failing with
>>>>>>>> "operation not supported".
>>>>>>>>
>>>>>>>> What do you mean by host "not running SELinux". SElinux is not
>>>>>>>> compiled
>>>>>>>> in? Or it is disabled or in passive mode?
>>>>>>>>
>>>>>>>> Is it working with filesystems other than btrfs, say ext4 or xfs.
>>>>>>>>
>>>>>>>> Now qemu supports xattr remapping. You might want to run virtiofsd
>>>>>>>> to remap security.selinux. I think that might get you going till
>>>>>>>> the root cause of the issue is found.
>>>>>>>>
>>>>>>>> Vivek
>>>>>>> Thank you for the focus.   The host os in this instance is not
>>>>>>> from the
>>>>>>> fedora/rhel/centos world with selinux running.  My case is a debian
>>>>>>> sourced distro (ubuntu).  That world uses 'apparmor' by default, not
>>>>>>> selinux.   I think it's reasonable to suppose there are a lot of
>>>>>>> servers
>>>>>>> out there not running selinux that have lots of vms running on
>>>>>>> them, not
>>>>>>> all using virtiofs.  There should be a documented way to allow the
>>>>>>> 'restorcon' command on one of many guests on such hosts to work.  I
>>>>>>> suppose to wrap this up:
>>>>>>>
>>>>>>> For the future readers who got here by searching,  could you give the
>>>>>>> first kernel version that supports a non-selinux host supporting an
>>>>>>> selinux enabled guest and the virtiofsd command line necessary to get
>>>>>>> the restorecon command to work normally?
>>>>>> I don't know yet. Because I don't know what's the root cause of the
>>>>>> issue.
>>>>>>
>>>>>> The way you are explaining it, looks like host kernel somehow is
>>>>>> blocking setxattr(security.selinux). And I have no idea why. Is it
>>>>>> apparmor or something else.
>>>>>>
>>>>>> If no selinux module is loaded on host, then as long as virtiofsd
>>>>>> process has CAP_SYS_ADMIN, it should be able to set security.selinux.
>>>>>>
>>>>>> "Operation not supported" means error "EOPNOTSUP". I am assuming
>>>>>> you are running virtiofsd with "-o xattr" to make sure virtiofsd
>>>>>> supports xattr. If that's the case somehow kernel is returning
>>>>>> "EOPNOTSUP".
>>>>>>
>>>>>> Can you run virtiofsd with debug option -d and try to install that
>>>>>> package in guest and capture outout of virtiofsd and post here. It
>>>>>> might confirm that host kernel is returning error.
>>>>> I tried doing "chcon unconfined_u:object_r:admin_home_t:s0 bar.txt"
>>>>> on a file in virtiofs and got "Operation not supported". I think
>>>>> guest kernel failed this. Will need to debug further.
>>>> Ok, Dan Walsh says that it probably is due to the fact that selinux
>>>> policy in guest is not aware of virtiofs. He has opened a PR to
>>>> add that.
>>>>
>>>> https://github.com/fedora-selinux/selinux-policy/pull/478
>>>>
>>>> I am not sure what distribution you are running as guest but it
>>>> probably will require similar changes. Once this package is built
>>>> I will give it a try.
>>>>
>>>> Thanks
>>>> Vivek
>>> Correct. The Guest OS Has to have SELinux enabled and the virtiofs
>>> file system within the VM
>>>
>>> needs to have SELinux policy that says it support labeling on Xattrs. 
>>> Otherwise when you attempt
>>>
>>> to set labels on the file system.  SELinux in side of the kernel will
>>> say that virtiofs does not support
>>>
>>> SELinux labels, which is what you are seeing.
>>>
>> It is the advertising and presumption of those using 'virtual machines'
>> that they are 'runnable' on any host.  If I read the above correctly,
>> because there's no telling which of the hundreds of packages in the
>> fedora/centos/rhel world will fail on built-in restorecon calls,
>> virtiofs is now excluded for general use except on SELinux enabled hosts
>> .
> Hi,
>
> This is SELinux policy change required in guest (and not host). So after
> this change in selinux policy in guest it should work in your setup
> (where you are not running SELinux on host). Can you please give it
> a try. selinux developers provided simple instructions to test it.
>
> https://github.com/fedora-selinux/selinux-policy/pull/478#issuecomment-731290656
>
> *********************
> FWIW, you can apply the fix locally without rebuilding the selinux-policy RPM as follows:
>
> # echo '(fsuse xattr virtiofs (system_u object_r fs_t ((s0) (s0))))' >virtiofs.cil
> # semodule -i virtiofs.cil
> And to check that the change is applied:
>
> # seinfo --fs_use | grep virtiofs
>     fs_use_xattr virtiofs system_u:object_r:fs_t:s0;
> To revert the local workaround:
>
> # semodule -r virtiofs
> ***********************************
>
> So please load above policy module in guest (and not host) and then try
> installing the package which was failing for you. Please let us know
> if this fixes the issue you are seeing or not. 
>
> I tested it and it fixed the chcon issue I was seeing.
>
>>  There are, (cough) a fair few hosts out there which are not running
>> SElinux, whose operators hope/need to provide vm guest services to the
>> fedora/rhel/centos package users.  So, I ask the virtiofs folks to
>> consider creating or defining an option allowing fedora/rhel/centos
>> guests a way to succeed.  Or, in the alternative, a clear warning that
>> virtiofs is not a good choice for  rhel/centos/fedora guests on other
>> than rhel/centos/fedora bare-metal requiring selinux enabled.
> To enable selinux in guest, we don't need selinux to be enabled 
> on host.
>
> In fact selinux policy on on host can potentially interfere with guest
> policy. So I think we should run virtiofsd with remapped
> "security.capability" xattr in qemu. That way both guest and host can
> have their own selinux policy.
>
> Thanks
> Vivek
>
Testing results follow.  Short version:  Commands above applied without
error,  failure remains until vm is rebooted, then success.  Good enough
for today!

Thanks

Harry Coin

---

  Detail:

In this case, the VM host is running a debian/ubuntu os, not running
selinux, the underlying filesystem is btrfs. 

root at noc1:~# uname -a
Linux noc1.1.quietfountain.com 5.8.0-29-generic #31-Ubuntu SMP Fri Nov 6
12:37:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

ps axu

...

root       84306  0.0  0.0  80216   992 ?        Sl   14:45   0:00
/usr/lib/qemu/virtiofsd --fd=44 -o
source=/vmsystems/fedora_generic,xattr,flock,no_posix_lock -o writeback

root       84356  4.1  0.0 4165836 30088 ?       Sl   14:45   1:23
/usr/lib/qemu/virtiofsd --fd=44 -o
source=/vmsystems/fedora_generic,xattr,flock,no_posix_lock -o writeback

...

On the otherwise default fedora workstation guest we have:

[root at fedora ~]# uname -a
Linux fedora.1.quietfountain.com 5.9.10-200.fc33.x86_64 #1 SMP Mon Nov
23 18:12:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root at fedora ~]# mount

...

myfs on / type virtiofs (rw,relatime)

...

[root at fedora ~]# touch foo
[root at fedora ~]# restorecon foo
restorecon: Could not set context for /root/foo:  Operation not supported
[root at fedora ~]# echo '(fsuse xattr virtiofs (system_u object_r fs_t
((s0) (s0))))' >virtiofs.cil
[root at fedora ~]# semodule -i virtiofs.cil
[root at fedora ~]# seinfo --fs_use | grep virtiofs

fs_use_xattr virtiofs system_u:object_r:fs_t:s0;

[root at fedora ~]# restorecon foo
restorecon: Could not set context for /root/foo:  Operation not supported
[root at fedora ~]# touch foo2
[root at fedora ~]# restorecon foo2
restorecon: Could not set context for /root/foo2:  Operation not supported
[root at fedora ~]# reboot

...

[root at fedora ~]# restorecon foo2
[root at fedora ~]# touch foo3
[root at fedora ~]# restorecon foo3
[root at fedora ~]#

More information about the Virtio-fs mailing list