[Pulp-dev] pulp 2 to 3 upgrade experience concerns

[Dennis Kliban]

After closer examination of the filesystem of a Pulp 2 install, I have
determined that we do not need to change the permissions of
/var/lib/pulp/content because the 'other' users can already read the files
stored there. That means we don't need to worry about long running scripts
to prepare the filesystem for migration to Pulp 3.

I have already closed a related issue[0]

I am going to update my PR[1] to reflect this. I will remove the -R from
the 'chown' command here[2] so the change is not performed recursively.

[0] https://pulp.plan.io/issues/5154#note-4
[1] https://github.com/pulp/pulp-packaging/pull/102
[2]
https://github.com/pulp/pulp-packaging/blob/master/packages/pulp/pulp.spec#L513

On Wed, Jul 31, 2019 at 12:18 PM Brian Bouterse <bmbouter at redhat.com> wrote:

>
>
> On Wed, Jul 31, 2019 at 12:12 PM Tatiana Tereshchenko <ttereshc at redhat.com>
> wrote:
>
>>
>>
>> On Wed, Jul 31, 2019 at 5:15 PM Dennis Kliban <dkliban at redhat.com> wrote:
>>
>>> On Wed, Jul 31, 2019 at 10:27 AM Brian Bouterse <bmbouter at redhat.com>
>>> wrote:
>>>
>>>> Thank you for raising this concern.
>>>>
>>>> On Wed, Jul 31, 2019 at 10:06 AM Dennis Kliban <dkliban at redhat.com>
>>>> wrote:
>>>>
>>>>> Users of Pulp 2 that are upgrading to Pulp 3 need to make files stored
>>>>> in /var/lib/pulp/content readable by Pulp 3. There are two way to do that:
>>>>>
>>>>> 1. Install Pulp 3 with 'pulp_user' set to 'apache'. (Pulp will run as
>>>>> user 'apache')
>>>>>
>>>>> 2. Install Pulp 3 with any 'pulp_user' and any 'pulp_group'.
>>>>>     Recursively change group ownership of /var/lib/pulp/content/ to
>>>>> the value of 'pulp_group' and change the mode to 775 so that group can read
>>>>> the files.
>>>>>
>>>>> We have already started moving users toward the second approach by
>>>>> including the relabeling of /var/lib/pulp in the 2.20.0 release[0].
>>>>> However, I am concerned that this will lead to poor experiences for users
>>>>> that have a lot of content in their Pulp 2. Hours or days of upgrading
>>>>> their Pulp to 2.20.0+.
>>>>>
>>>> I hear this concern.
>>>>
>>>>
>>>>> I propose that we revert the change introduces in 2.20.0 and add an
>>>>> RPM scriptlet that changes everything back to 'apache:apache'. Then we can
>>>>> tell users that they have the two options from above when upgrading from
>>>>> Pulp 2.
>>>>>
>>>> The installer could be configured to work with these requirements It
>>>> takes those options currently AIUI. This proposal doesn't adjust the
>>>> installer's defaults does it?
>>>>
>>>>>
>>>>>
>>> The installer only supports 'pulp_user'. The 'pulp_group' is being
>>> introduced in an open PR[0].
>>>
>>> [0] https://github.com/pulp/ansible-pulp/pull/128/files
>>>
>> Great thank you.
>
>
>>>
>>>
>>>> What are your thoughts?
>>>>>
>>>> I agree it takes time to perform that permission updating. The
>>>> motivation was that we didn't want Pulp as an application running as user
>>>> apache, especially with apache becoming a reverse proxy for Pulp3. We can
>>>> rethink that, but that is the current prevailing logic that I think of. I'm
>>>> open to other ideas here. If this is the case, and the filesystem is large,
>>>> taking time to fix these permissions is unavoidable (to me). The best thing
>>>> we can do is ensure it can be done online.
>>>>
>>>> What if we had better alternatives to alleviate the long-running rpm
>>>> upgrade? We could make a script that adjusts the groups so that either pulp
>>>> or apache could be the owners, and the performs the relabeling while pulp2
>>>> 2.19.z is online before the 2.20 upgrade. This would move the workload out
>>>> of upgrade time during downtime and to a pre-migration uptime. What about
>>>> ideas like that?
>>>>
>>>>
>>> I like the idea of providing a script that will run while Pulp 2 is
>>> operating. However, we should probably still change the behavior of Pulp 2
>>> so that it starts creating new files with the correct permissions. This way
>>> we can guarantee that the filesystem relabeling will eventually finish.
>>> Pulp 2 workers receive these settings through systemd unit files. The
>>> desired user and group values will need to be specified in those systemd
>>> unit file for Pulp 2. The script that does the relabeling will need to use
>>> these same values.
>>>
>> Agreed, we need Pup2 to be writing as the new user. If the posix user and
> groups are right Pulp2 should be able to do this. Also maybe we have the
> setuid bit perhaps so it sets the uid for new files and Pulp doesn't have
> to worry about it? Is that a possibility?
>
>>
>> How to ensure that the relabeling is fully done before upgrade to Pulp 3?
>>
> What about if the migration tool performs a check-and-fix or just a
> check-and-report for the user to run a script again to fix.
>
>>
>>
>>
>>>
>>>
>>>> One more thought, this is specifically slow in NFS environments, we
>>>> could have users set the uid, gid on the NFS server itself.
>>>>
>>>>
>>> The gid needs to match on both machines. The pulp-server RPM for Pulp 2
>>> needs to set the gid when creating the group and ansible-pulp will need set
>>> the gid to the same value when creating the group.
>>>
>> The matching of uid/gid is pretty typical NFS stuff, so I think this is
> ok.
>
>>
>>>
>>>
>>>>
>>>>>
>>>>> [0]  https://github.com/pulp/pulp-packaging/pull/98/files
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190801/2d98d708/attachment.htm>