[Pulp-dev] RBAC Status Thread

Brian Bouterse bmbouter at redhat.com
Thu Aug 6 18:49:10 UTC 2020

# Update:

django-lifecycle merged + released, and all tests are passing on Travis
now  https://github.com/pulp/pulpcore/pull/815/files

# Regarding assigning permissions to objects created prior to RBAC...

I now believe we do not need to apply permissions for the 'admin' user, and
that is the only user we could expect at migration time because it's the
only user we deal with prior to 3.6. I've learned that by default,
django-guardian considers any user with `is_superuser=True` to have the
permission being checked. As such the 'admin' user already sidesteps RBAC
anyway. So my next step in this area is to add this to the docs only.
Plugin writers will *not* need to add permissions for prior existing

On Wed, Aug 5, 2020 at 5:59 PM Brian Bouterse <bmbouter at redhat.com> wrote:

> Alright we're close to the end of pulpcore's RBAC additions for 3.6, but
> there are still some challanges! Here's what's been accomplished:
> * The tests are now passing locally
>   * We are waiting on django-lifecycle to merge my PR and release it
> https://github.com/rsinger86/django-lifecycle/pull/58 We actually can't
> release on my fork (PyPI install_requires won't work with git checkouts
> I've learned) so if he can't release django-lifecycle by Friday then I'm
> going to push an alternate package to PyPI in the meantime under a
> different name. This is what #python recommended to me. I have emailed the
> author asking for review and release so I'm hoping it does not come to that
> * lots more docs have been written
> * I can't show the tests passing on Travis until this django-lifecycle
> issue is resolved, so I'm going to keep the WIP on it until then.
> User docs are still needed, I'm working on those tomorrow
> Please do review, I want to merge on Friday to be included in 3.6.
> On Fri, Jul 31, 2020 at 4:58 PM Brian Bouterse <bmbouter at redhat.com>
> wrote:
>> A lot of plugin writer docs about adding RBAC support have been added to
>> the newly opened WIP PR: https://github.com/pulp/pulpcore/pull/815/files
>> Please come review the WIP PR. Code-wise it's 90% there, with just a few
>> polish things I still need to finish. From a high level I still need to:
>> * finish the plugin writer docs
>> * add users docs (smaller and easier than plugin writer docs)
>> * ensure all the changelogs and issues are good
>> * get Tasks showing with their object-level permissions in the
>> Django-admin
>> * fix the tests so they are passing on Travis
>> The plan is to have this go into 3.6 to be released Aug 11th.
>> The other big area of work (also for 3.6) is that galaxy_ng is needing
>> Groups (and because of that Users) APIs added for read-only. Those stores
>> got a lot of grooming today and are linked to below. @fabricio is taking
>> the lead on implementing that. Thanks @fabricio!
>> https://pulp.plan.io/issues/7231
>> https://pulp.plan.io/issues/7232
>> Feedback and other concerns are welcome.
>> On Fri, Jul 24, 2020 at 4:58 PM Brian Bouterse <bmbouter at redhat.com>
>> wrote:
>>> I've gotten various feedback and I want to relay some of the changes
>>> based on that.
>>> 1) Pulp should ship a "user isolation" policy by default. If users want
>>> other things they can configure it further. This is a change from my
>>> original proposal of Pulp shipping an "RBAC is off" policy. This will work
>>> for Katello because all Katello calls show up as one user anyway.
>>> 2) We have challenges around assigning permissions. This is the proposed
>>> solution that was discussed at open floor today:
>>> https://pulp.plan.io/issues/7210  I've added this to my pulpcore PR
>>> (link below)
>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>> On Fri, Jul 10, 2020 at 2:20 PM David Davis <daviddavis at redhat.com>
>>> wrote:
>>>> On Wed, Jul 8, 2020 at 4:54 PM Brian Bouterse <bmbouter at redhat.com>
>>>> wrote:
>>>>> My next goal is to have object-level permissions assigned through
>>>>> signals so that anywhere you save the model the permissions are correctly
>>>>> created. To do this I need to get a few things working:
>>>>> 1) Move the permissions creation to the signals [done]
>>>>> 2) Have the user be well-known, this is accomplished by pulpcore
>>>>> picking up this dependency
>>>>> https://github.com/PaesslerAG/django-currentuser [done]
>>>>> 3) Have the user information persisted in tasks so signals can work
>>>>> there also ... there are two obvious options
>>>>> option a) Have tasks themselves have RBAC which we know we need
>>>>> anyway, then query the RBAC permissions inside a task to determine which
>>>>> user has the object level permissions for this task. This is what I'm
>>>>> currently prototyping. It at least will provide a mechanism we can use for
>>>>> now until we find something better.
>>>> Are task permissions mutable? If so, then what happens if someone kicks
>>>> off a task and then this permission gets removed?
>>>>> option b) Add a user field to the Task itself and have the RBAC assign
>>>>> permissions to them. The concern with this is that while this is nice, it
>>>>> would be nicer if we had this type of visibility on everything in Pulp from
>>>>> an auditing project dedicated to keeping this type of info. Option (b) is
>>>>> still viable, just maybe not the best way. Comments/feedback is welcome.
>>>>> Also I've collaborated with @alikins from galaxy_ng project and
>>>>> together we wrote this requirements doc for their plugin:
>>>>> https://hackmd.io/JisLkfyeT2myAD2khEwU0A
>>>>> On Wed, Jul 1, 2020 at 6:43 PM Brian Bouterse <bmbouter at redhat.com>
>>>>> wrote:
>>>>>> The demo advertisement for tomorrow is here:
>>>>>> https://www.redhat.com/archives/pulp-dev/2020-June/msg00076.html
>>>>>> On Wed, Jul 1, 2020 at 6:41 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>> wrote:
>>>>>>> Another productive RBAC day! See the latest code at the links below.
>>>>>>> Here's what's new:
>>>>>>> * policy is now shorter thanks to machinery checking both
>>>>>>> model-level and object-level permissions with one call. The other two are
>>>>>>> also available
>>>>>>> * sync is now restricted on both 'modify_repo_content' permissions
>>>>>>> AND read permission on the remote being used to sync
>>>>>>> * modify is now restricted on 'modify_repo_content' permission
>>>>>>> * moved the permission checking machinery to be "global checks"
>>>>>>> * added data migration that sets is_staff=True, so the django-admin
>>>>>>> interface can be used (this is getting a slight rework tomorrow morning tho)
>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC
>>>>>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>>>>>> Tomorrow's demo is advertised here. It will also include an overview
>>>>>>> of some of the unsolved problems with some possible solutions.
>>>>>>> Cheers,
>>>>>>> Brian
>>>>>>> On Tue, Jun 30, 2020 at 5:08 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>>> wrote:
>>>>>>>> Today I accomplished a few more things:
>>>>>>>> * finished my ldap notes: https://hackmd.io/ED9UpscNSRW86Le3xNzVeg
>>>>>>>> * moving the checks from a mixin to be "global checks" so they are
>>>>>>>> available everywhere, this is a feature from drf-access-policy:
>>>>>>>> https://rsinger86.github.io/drf-access-policy/reusable_conditions/
>>>>>>>> * added a has_obj_or_module_perms method allowing policy writers to
>>>>>>>> just use that instad of carrying "two entries" in the policy, one for
>>>>>>>> model-level, one for object-level
>>>>>>>> Need to:
>>>>>>>> * clean up the "sync" policy code
>>>>>>>> * Add global condition check facilities for the perms of a 'remote'
>>>>>>>> param
>>>>>>>> * add policy language restricting the /modify/ endpoint also for
>>>>>>>> FileRepository
>>>>>>>> * push my code
>>>>>>>> New Challenge: We need to also have the permissions assignments
>>>>>>>> happen for objects created by tasks. django-guardian recommends this happen
>>>>>>>> inside signals (
>>>>>>>> https://django-guardian.readthedocs.io/en/latest/userguide/assign.html#assigning-permissions-inside-signals).
>>>>>>>> The challenge (thanks @mdellweg for identifying) is that the user/group
>>>>>>>> context information is well-known in the viewset but not in a task. Soooooo
>>>>>>>> ... the idea is:
>>>>>>>> 1. Switch the perms addition to the model itself via signals so
>>>>>>>> it's automatic everywhere (including in tasks)
>>>>>>>> 2. Preserve the user and group "request context" into the tasking
>>>>>>>> system. I can see a straightforward path to how to do this so I plan to
>>>>>>>> prototype this soon also.
>>>>>>>> Feedback is welcome!
>>>>>>>> On Fri, Jun 26, 2020 at 6:16 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>>>> wrote:
>>>>>>>>> Today I got the "sync" RBAC working, but I need to give it some
>>>>>>>>> more thought. The extra challenge with this parts is that "having
>>>>>>>>> permission to read a Remote" is already defined in one place, on
>>>>>>>>> FileRemoteAccessPolicy, yet the AccessPolicy that needs to perform the
>>>>>>>>> enforcement is FileRepositoryAccessPolicy for its "sync" action. This is a
>>>>>>>>> bit challenging considering the following goals:
>>>>>>>>> * We don't want to duplicate code, e.g. having the
>>>>>>>>> FileRepositoryAccessControl begin to inspect permissions for FileRemote
>>>>>>>>> directly, when FileRemoteAccessPolicy already does that
>>>>>>>>> * Currently permissions are granted at two levels: Model-level and
>>>>>>>>> File-level permissions and permissions are granted from either level.
>>>>>>>>> * We want to keep the policy in charge. If we start to bury the
>>>>>>>>> behavior in methods and functions then policy writers are no longer in
>>>>>>>>> control.
>>>>>>>>> All of ^ together tells me that I should work on creating two
>>>>>>>>> things next:
>>>>>>>>> 1) A way for policy writers to express which parameter refers to
>>>>>>>>> objects that also need their permissions checked. For example the policy
>>>>>>>>> should be able to say "remote is a parameter and it needs X permission".
>>>>>>>>> This is akin to the has_module_level_perms and has_obj_level_perms here
>>>>>>>>> except we also need to identify which parameter is being checked instead of
>>>>>>>>> the object the AccessPolicy itself governs.
>>>>>>>>> 2) A single way to check model-level and object-level permissions
>>>>>>>>> at once and allow if *either* passes. We would still allow policy writers
>>>>>>>>> to call either model-level or file-level checks also.
>>>>>>>>> I'll work on ^ next. Ideas and feedback are welcome. I pushed no
>>>>>>>>> new code today because it's a mess and not runnable at my stopping point.
>>>>>>>>> On Thu, Jun 25, 2020 at 6:18 PM Brian Bouterse <
>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>> Here's another push to the branch (it includes the following
>>>>>>>>>> additions):
>>>>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>>>> * A FileRepositoryAccessPolicy which provides RBAC for
>>>>>>>>>> Repositories (not yet sync)
>>>>>>>>>> * A new Mixin allowing the two policies to share some common
>>>>>>>>>> components
>>>>>>>>>> Next up:
>>>>>>>>>> * have the pup_file define the fileContentAdmin group
>>>>>>>>>> programmatically
>>>>>>>>>> * Extend the FileRepositoryAccessPolicy to restrict sync
>>>>>>>>>> operations
>>>>>>>>>> * Write up and organize the PoC into a clear, organized format
>>>>>>>>>> Also of interest today @ttereshc and I had a great convo asking
>>>>>>>>>> what to do about potential problems when we use Django groups to be a
>>>>>>>>>> "role". My write up will address this in more detail than I can go into
>>>>>>>>>> here. We are also looking at what the django-role-permissions project could
>>>>>>>>>> offer us:
>>>>>>>>>> https://django-role-permissions.readthedocs.io/en/stable/utils.html
>>>>>>>>>> I expect the PoC to be done by tomorrow and write-up by Monday,
>>>>>>>>>> so I'm going to schedule the public review meeting for next week towards
>>>>>>>>>> the end of the week.
>>>>>>>>>> On Wed, Jun 24, 2020 at 5:49 PM Brian Bouterse <
>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>> Moar progress! Today the following things got done: Today's
>>>>>>>>>>> changes are available here:
>>>>>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>>>>> * Got scoped querysets working! This restricts list views to
>>>>>>>>>>> only show objects a user has permissions to view. A db reset was all that
>>>>>>>>>>> was needed I think I didn't have all the changes in when I applied my
>>>>>>>>>>> earlier migrations
>>>>>>>>>>> * Added "detail view" restriction, and while it's in the policy
>>>>>>>>>>> and working DRF does a strange thing on "retrieve" where if it's not in the
>>>>>>>>>>> queryset (due to scoping ^) the user receives a 404, not a permission denied
>>>>>>>>>>> * Got permissions cleaning up on resource deletion now too
>>>>>>>>>>> Next up:
>>>>>>>>>>> * have the pup_file define the fileContentAdmin group
>>>>>>>>>>> programmatically
>>>>>>>>>>> * Make similar policies for FileRepository which governs itself
>>>>>>>>>>> and the "sync" action
>>>>>>>>>>> * Write up and organize the PoC into a clear, organized format
>>>>>>>>>>> Questions and feedback are welcome!
>>>>>>>>>>> On Tue, Jun 23, 2020 at 5:54 PM Brian Bouterse <
>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>> Lots of progress today! I have a mostly-complete policy for
>>>>>>>>>>>> RBAC for FileRemote. It's surprising how little code all of this ended up
>>>>>>>>>>>> being.
>>>>>>>>>>>> Here's the actual RBAC stuff, it's all in pulp_file:
>>>>>>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>>>>>> Here's the parts that go in core. Note the LDAP stuff is all
>>>>>>>>>>>> optional, the only real requirement are two lines 1) enabling guardian in
>>>>>>>>>>>> INSTALLED_APPS and 2) adding it as an AuthenticationBackend:
>>>>>>>>>>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>>>>>>>>>>> I have some "how to use notes" here:
>>>>>>>>>>>> https://hackmd.io/DRqGFyRsSDmN7E4TtOPf-w  The idea is that it
>>>>>>>>>>>> implements the FileRemote portions of this requirements docs:
>>>>>>>>>>>> https://hackmd.io/kZ1oYp8TTkeuC5KL_ffjGQ
>>>>>>>>>>>> Here is the short list of things for FileRemote that still
>>>>>>>>>>>> don't work. This is mainly so I remember what to do next. :)
>>>>>>>>>>>> * The get_objects_for_user
>>>>>>>>>>>> <https://django-guardian.readthedocs.io/en/latest/api/guardian.shortcuts.html#get-objects-for-user>
>>>>>>>>>>>> from DjangoGuardian I don't think it likes Master/Detail or maybe it's
>>>>>>>>>>>> how/where I'm using it. I haven't yet debugged this. For this reason it
>>>>>>>>>>>> doesn't provide list restriction
>>>>>>>>>>>> * It still needs "detail view" restriction. This is
>>>>>>>>>>>> straightforward.
>>>>>>>>>>>> * The group should be programmatically defined, in this case it
>>>>>>>>>>>> was "defined in LDAP". It could *also* live in LDAP (or other external
>>>>>>>>>>>> group definition system) but the plugin builds permissions off of it so it
>>>>>>>>>>>> should also define it. This is easy.
>>>>>>>>>>>> Feedback is welcome. I'm going to continue building this and
>>>>>>>>>>>> then schedule a public review of FileRemote, Content modification for file
>>>>>>>>>>>> repos, and sync restriction next week.
>>>>>>>>>>>> On Mon, Jun 22, 2020 at 5:14 PM Brian Bouterse <
>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>> # ldap PoC updates
>>>>>>>>>>>>> Now users, groups, and group membership are populating from
>>>>>>>>>>>>> ldap automatically on login (with auth backed by ldap also)! I'll be
>>>>>>>>>>>>> sharing my configs for both ldap and how to configure
>>>>>>>>>>>>> django-auth-ldap
>>>>>>>>>>>>> <https://django-auth-ldap.readthedocs.io/en/latest/example.html>
>>>>>>>>>>>>> here soon in an organized way. This was done with django-auth-ldap and 0
>>>>>>>>>>>>> customization to pulp code. It's 100% enabled through settings so this work
>>>>>>>>>>>>> is more of an approach we can document for users that they can enable and
>>>>>>>>>>>>> not a feature Pulp ships itself.
>>>>>>>>>>>>> # django-admin progress
>>>>>>>>>>>>> Thanks to @alikins existing PRs, I got django admin enabled
>>>>>>>>>>>>> and able to view/edit users, groups, group membership, and permissions at
>>>>>>>>>>>>> both the user and group levels. This is important because this will be the
>>>>>>>>>>>>> primary mechanism of administrators. This part is looking good.
>>>>>>>>>>>>> # new resources to help us out
>>>>>>>>>>>>> Through collaboration with @ttereshc and someone off list
>>>>>>>>>>>>> named @adelton (who actually authored this reference approach
>>>>>>>>>>>>> <https://www.adelton.com/django/external-authentication-for-django-projects>
>>>>>>>>>>>>> I referenced early on in this exploration), this very cool repository of
>>>>>>>>>>>>> testing tools was identified:
>>>>>>>>>>>>> https://github.com/adelton/webauthinfra  It has a treasure
>>>>>>>>>>>>> trove of testing containers which Pulp devs in the future can test against.
>>>>>>>>>>>>> It keeps the user/group check in the apache which is fine alternative to
>>>>>>>>>>>>> the django-auth-ldap approach above. Pulp doesn't have to choose, it could
>>>>>>>>>>>>> work with either just configured differently. The pending PoC outline will
>>>>>>>>>>>>> go over these alternative approaches in detail.
>>>>>>>>>>>>> # Next Steps:  back to the PoC itself
>>>>>>>>>>>>> Now that we have demonstrated good options of external
>>>>>>>>>>>>> users/groups/membership loading into Pulp we can confidently move back to
>>>>>>>>>>>>> finishing the RBAC PoC itself. I've started back into this. So the
>>>>>>>>>>>>> remaining work are the two steps below:
>>>>>>>>>>>>> 1. Finish the PoC that uses RBAC to restrict remotes, sync,
>>>>>>>>>>>>> and repository content modification. Currently I prototyped restriction of
>>>>>>>>>>>>> operations on Remotes, but I need to replicate the access policies to
>>>>>>>>>>>>> Repositories and Sync next.
>>>>>>>>>>>>> 2. Write it up and share it.
>>>>>>>>>>>>> 3. Schedule public meeting to review it (targeting next-week)
>>>>>>>>>>>>> On Fri, Jun 19, 2020 at 5:09 PM Brian Bouterse <
>>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>> I got the LDAP users both authenticating and importing into
>>>>>>>>>>>>>> Pulp! Next I'll do the groups and then I think the ldap parts will be done.
>>>>>>>>>>>>>> FYI: I'm going to write up the implementation design and have
>>>>>>>>>>>>>> that come with this proof of concept code . This will let us know what
>>>>>>>>>>>>>> choices it makes, why it makes them, and we can determine if these are the
>>>>>>>>>>>>>> right choices together.
>>>>>>>>>>>>>> On Wed, Jun 17, 2020 at 4:57 PM Brian Bouterse <
>>>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>>> I got a lot further on this today. I have the test ldap
>>>>>>>>>>>>>>> setup with several test users and groups. I have django-auth-ldap
>>>>>>>>>>>>>>> configured mostly authenticating username/password against ldap instead of
>>>>>>>>>>>>>>> the internal database first. Once that is fully working the users will
>>>>>>>>>>>>>>> auto-populate into django and the groups should follow easily.
>>>>>>>>>>>>>>> Once that's done I'll be unblocked to finish the RBAC PoC.
>>>>>>>>>>>>>>> The rest of the parts are straightforward given the testing I've already
>>>>>>>>>>>>>>> done. More updates to come.
>>>>>>>>>>>>>>> On Mon, Jun 15, 2020 at 5:03 PM Brian Bouterse <
>>>>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>>>> I got the ldap reference implementation performing auth
>>>>>>>>>>>>>>>> really nicely against a test ldap with this guide:
>>>>>>>>>>>>>>>> https://www.nginx.com/blog/nginx-plus-authenticate-users/
>>>>>>>>>>>>>>>> Now there are some new challenges though:
>>>>>>>>>>>>>>>> * Great that we can auth users, but we need nginx to
>>>>>>>>>>>>>>>> extract-and-forward the group information to Pulp itself. That way a
>>>>>>>>>>>>>>>> middleware can create the user AND group info in the backend.
>>>>>>>>>>>>>>>> * we have to figure this out all again in Apache...
>>>>>>>>>>>>>>>> Maybe we should be integrating Pulp directly against
>>>>>>>>>>>>>>>> django-auth-ldap [0]. I am going to try that next. The work I've done isn't
>>>>>>>>>>>>>>>> 100% reusable there, but most of it is because the test server and configs
>>>>>>>>>>>>>>>> I used can all be reused directly with django-auth-ldap. The concern with
>>>>>>>>>>>>>>>> this approach is that we would be supporting LDAP (and transitively Active
>>>>>>>>>>>>>>>> Directory) but are there other directory services Pulp needs to support?
>>>>>>>>>>>>>>>> I also emailed Bin Li asking for info on how their user and
>>>>>>>>>>>>>>>> group management works.
>>>>>>>>>>>>>>>> On Tue, Jun 9, 2020 at 11:48 AM Adrian Likins <
>>>>>>>>>>>>>>>> alikins at redhat.com> wrote:
>>>>>>>>>>>>>>>>> On Fri, Jun 5, 2020 at 8:23 PM Brian Bouterse <
>>>>>>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>>>>>> 1) django admin (the built in django UI) will be the
>>>>>>>>>>>>>>>>>> mechanism administrators use to assign permissions to users and groups.
>>>>>>>>>>>>>>>>>> This means the use of django admin with pulp is very likely (to me).
>>>>>>>>>>>>>>>>>> Hopefully https://github.com/pulp/pulpcore/pull/705 will
>>>>>>>>>>>>>>>>> be useful here.
>>>>>>>>>>>>>>>>>> 2) externally defined users and groups will need to be
>>>>>>>>>>>>>>>>>> "replicated" to django's db at login time, probably using headers from the
>>>>>>>>>>>>>>>>>> webserver This is consistent w/ the approach recommended here:
>>>>>>>>>>>>>>>>>> https://www.adelton.com/django/external-authentication-for-django-projects
>>>>>>>>>>>>>>>>> This is more or less what galaxy_ng ends up doing, at
>>>>>>>>>>>>>>>>> least for the scenarios where it runs hosted with external SSO.
>>>>>>>>>>>>>>>>> https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/auth/auth.py#L51 for
>>>>>>>>>>>>>>>>> example.
>>>>>>>>>>>>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200806/5022b334/attachment.htm>

More information about the Pulp-dev mailing list