[Pulp-dev] RBAC Status Thread

Wed Jul 1 22:43:02 UTC 2020

The demo advertisement for tomorrow is here:
https://www.redhat.com/archives/pulp-dev/2020-June/msg00076.html

On Wed, Jul 1, 2020 at 6:41 PM Brian Bouterse <bmbouter at redhat.com> wrote:

> Another productive RBAC day! See the latest code at the links below.
> Here's what's new:
>
> * policy is now shorter thanks to machinery checking both model-level and
> object-level permissions with one call. The other two are also available
> * sync is now restricted on both 'modify_repo_content' permissions AND
> read permission on the remote being used to sync
> * modify is now restricted on 'modify_repo_content' permission
> * moved the permission checking machinery to be "global checks"
> * added data migration that sets is_staff=True, so the django-admin
> interface can be used (this is getting a slight rework tomorrow morning tho)
>
> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC
> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>
> Tomorrow's demo is advertised here. It will also include an overview of
> some of the unsolved problems with some possible solutions.
>
> Cheers,
> Brian
>
>
> On Tue, Jun 30, 2020 at 5:08 PM Brian Bouterse <bmbouter at redhat.com>
> wrote:
>
>> Today I accomplished a few more things:
>>
>> * finished my ldap notes: https://hackmd.io/ED9UpscNSRW86Le3xNzVeg
>> * moving the checks from a mixin to be "global checks" so they are
>> available everywhere, this is a feature from drf-access-policy:
>> https://rsinger86.github.io/drf-access-policy/reusable_conditions/
>> * added a has_obj_or_module_perms method allowing policy writers to just
>> use that instad of carrying "two entries" in the policy, one for
>> model-level, one for object-level
>>
>> Need to:
>> * clean up the "sync" policy code
>> * Add global condition check facilities for the perms of a 'remote' param
>> * add policy language restricting the /modify/ endpoint also for
>> FileRepository
>> * push my code
>>
>> New Challenge: We need to also have the permissions assignments happen
>> for objects created by tasks. django-guardian recommends this happen inside
>> signals (
>> https://django-guardian.readthedocs.io/en/latest/userguide/assign.html#assigning-permissions-inside-signals).
>> The challenge (thanks @mdellweg for identifying) is that the user/group
>> context information is well-known in the viewset but not in a task. Soooooo
>> ... the idea is:
>>
>> 1. Switch the perms addition to the model itself via signals so it's
>> automatic everywhere (including in tasks)
>> 2. Preserve the user and group "request context" into the tasking system.
>> I can see a straightforward path to how to do this so I plan to prototype
>> this soon also.
>>
>> Feedback is welcome!
>>
>>
>>
>>
>>
>> On Fri, Jun 26, 2020 at 6:16 PM Brian Bouterse <bmbouter at redhat.com>
>> wrote:
>>
>>> Today I got the "sync" RBAC working, but I need to give it some more
>>> thought. The extra challenge with this parts is that "having permission to
>>> read a Remote" is already defined in one place, on FileRemoteAccessPolicy,
>>> yet the AccessPolicy that needs to perform the enforcement is
>>> FileRepositoryAccessPolicy for its "sync" action. This is a bit challenging
>>> considering the following goals:
>>>
>>> * We don't want to duplicate code, e.g. having the
>>> FileRepositoryAccessControl begin to inspect permissions for FileRemote
>>> directly, when FileRemoteAccessPolicy already does that
>>> * Currently permissions are granted at two levels: Model-level and
>>> File-level permissions and permissions are granted from either level.
>>> * We want to keep the policy in charge. If we start to bury the behavior
>>> in methods and functions then policy writers are no longer in control.
>>>
>>> All of ^ together tells me that I should work on creating two things
>>> next:
>>> 1) A way for policy writers to express which parameter refers to objects
>>> that also need their permissions checked. For example the policy should be
>>> able to say "remote is a parameter and it needs X permission". This is akin
>>> to the has_module_level_perms and has_obj_level_perms here except we also
>>> need to identify which parameter is being checked instead of the object the
>>> AccessPolicy itself governs.
>>> 2) A single way to check model-level and object-level permissions at
>>> once and allow if *either* passes. We would still allow policy writers to
>>> call either model-level or file-level checks also.
>>>
>>> I'll work on ^ next. Ideas and feedback are welcome. I pushed no new
>>> code today because it's a mess and not runnable at my stopping point.
>>>
>>>
>>> On Thu, Jun 25, 2020 at 6:18 PM Brian Bouterse <bmbouter at redhat.com>
>>> wrote:
>>>
>>>> Here's another push to the branch (it includes the following
>>>> additions):
>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>
>>>> * A FileRepositoryAccessPolicy which provides RBAC for Repositories
>>>> (not yet sync)
>>>> * A new Mixin allowing the two policies to share some common components
>>>>
>>>> Next up:
>>>> * have the pup_file define the fileContentAdmin group programmatically
>>>> * Extend the FileRepositoryAccessPolicy to restrict sync operations
>>>> * Write up and organize the PoC into a clear, organized format
>>>>
>>>> Also of interest today @ttereshc and I had a great convo asking what to
>>>> do about potential problems when we use Django groups to be a "role". My
>>>> write up will address this in more detail than I can go into here. We are
>>>> also looking at what the django-role-permissions project could offer us:
>>>> https://django-role-permissions.readthedocs.io/en/stable/utils.html
>>>>
>>>> I expect the PoC to be done by tomorrow and write-up by Monday, so I'm
>>>> going to schedule the public review meeting for next week towards the end
>>>> of the week.
>>>>
>>>>
>>>> On Wed, Jun 24, 2020 at 5:49 PM Brian Bouterse <bmbouter at redhat.com>
>>>> wrote:
>>>>
>>>>> Moar progress! Today the following things got done: Today's changes
>>>>> are available here:
>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>
>>>>> * Got scoped querysets working! This restricts list views to only show
>>>>> objects a user has permissions to view. A db reset was all that was needed
>>>>> I think I didn't have all the changes in when I applied my earlier
>>>>> migrations
>>>>> * Added "detail view" restriction, and while it's in the policy and
>>>>> working DRF does a strange thing on "retrieve" where if it's not in the
>>>>> queryset (due to scoping ^) the user receives a 404, not a permission denied
>>>>> * Got permissions cleaning up on resource deletion now too
>>>>>
>>>>> Next up:
>>>>> * have the pup_file define the fileContentAdmin group programmatically
>>>>> * Make similar policies for FileRepository which governs itself and
>>>>> the "sync" action
>>>>> * Write up and organize the PoC into a clear, organized format
>>>>>
>>>>> Questions and feedback are welcome!
>>>>>
>>>>> On Tue, Jun 23, 2020 at 5:54 PM Brian Bouterse <bmbouter at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Lots of progress today! I have a mostly-complete policy for RBAC for
>>>>>> FileRemote. It's surprising how little code all of this ended up being.
>>>>>>
>>>>>> Here's the actual RBAC stuff, it's all in pulp_file:
>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>> Here's the parts that go in core. Note the LDAP stuff is all
>>>>>> optional, the only real requirement are two lines 1) enabling guardian in
>>>>>> INSTALLED_APPS and 2) adding it as an AuthenticationBackend:
>>>>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>>>>>
>>>>>> I have some "how to use notes" here:
>>>>>> https://hackmd.io/DRqGFyRsSDmN7E4TtOPf-w  The idea is that it
>>>>>> implements the FileRemote portions of this requirements docs:
>>>>>> https://hackmd.io/kZ1oYp8TTkeuC5KL_ffjGQ
>>>>>>
>>>>>> Here is the short list of things for FileRemote that still don't
>>>>>> work. This is mainly so I remember what to do next. :)
>>>>>> * The get_objects_for_user
>>>>>> <https://django-guardian.readthedocs.io/en/latest/api/guardian.shortcuts.html#get-objects-for-user>
>>>>>> from DjangoGuardian I don't think it likes Master/Detail or maybe it's
>>>>>> how/where I'm using it. I haven't yet debugged this. For this reason it
>>>>>> doesn't provide list restriction
>>>>>> * It still needs "detail view" restriction. This is straightforward.
>>>>>> * The group should be programmatically defined, in this case it was
>>>>>> "defined in LDAP". It could *also* live in LDAP (or other external group
>>>>>> definition system) but the plugin builds permissions off of it so it should
>>>>>> also define it. This is easy.
>>>>>>
>>>>>> Feedback is welcome. I'm going to continue building this and then
>>>>>> schedule a public review of FileRemote, Content modification for file
>>>>>> repos, and sync restriction next week.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 22, 2020 at 5:14 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> # ldap PoC updates
>>>>>>> Now users, groups, and group membership are populating from ldap
>>>>>>> automatically on login (with auth backed by ldap also)! I'll be sharing my
>>>>>>> configs for both ldap and how to configure django-auth-ldap
>>>>>>> <https://django-auth-ldap.readthedocs.io/en/latest/example.html>
>>>>>>> here soon in an organized way. This was done with django-auth-ldap and 0
>>>>>>> customization to pulp code. It's 100% enabled through settings so this work
>>>>>>> is more of an approach we can document for users that they can enable and
>>>>>>> not a feature Pulp ships itself.
>>>>>>>
>>>>>>> # django-admin progress
>>>>>>> Thanks to @alikins existing PRs, I got django admin enabled and able
>>>>>>> to view/edit users, groups, group membership, and permissions at both the
>>>>>>> user and group levels. This is important because this will be the primary
>>>>>>> mechanism of administrators. This part is looking good.
>>>>>>>
>>>>>>> # new resources to help us out
>>>>>>> Through collaboration with @ttereshc and someone off list named
>>>>>>> @adelton (who actually authored this reference approach
>>>>>>> <https://www.adelton.com/django/external-authentication-for-django-projects>
>>>>>>> I referenced early on in this exploration), this very cool repository of
>>>>>>> testing tools was identified:
>>>>>>> https://github.com/adelton/webauthinfra  It has a treasure trove of
>>>>>>> testing containers which Pulp devs in the future can test against. It keeps
>>>>>>> the user/group check in the apache which is fine alternative to the
>>>>>>> django-auth-ldap approach above. Pulp doesn't have to choose, it could work
>>>>>>> with either just configured differently. The pending PoC outline will go
>>>>>>> over these alternative approaches in detail.
>>>>>>>
>>>>>>> # Next Steps:  back to the PoC itself
>>>>>>> Now that we have demonstrated good options of external
>>>>>>> users/groups/membership loading into Pulp we can confidently move back to
>>>>>>> finishing the RBAC PoC itself. I've started back into this. So the
>>>>>>> remaining work are the two steps below:
>>>>>>>
>>>>>>> 1. Finish the PoC that uses RBAC to restrict remotes, sync, and
>>>>>>> repository content modification. Currently I prototyped restriction of
>>>>>>> operations on Remotes, but I need to replicate the access policies to
>>>>>>> Repositories and Sync next.
>>>>>>> 2. Write it up and share it.
>>>>>>> 3. Schedule public meeting to review it (targeting next-week)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jun 19, 2020 at 5:09 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I got the LDAP users both authenticating and importing into Pulp!
>>>>>>>> Next I'll do the groups and then I think the ldap parts will be done.
>>>>>>>>
>>>>>>>> FYI: I'm going to write up the implementation design and have that
>>>>>>>> come with this proof of concept code . This will let us know what choices
>>>>>>>> it makes, why it makes them, and we can determine if these are the right
>>>>>>>> choices together.
>>>>>>>>
>>>>>>>> On Wed, Jun 17, 2020 at 4:57 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I got a lot further on this today. I have the test ldap setup with
>>>>>>>>> several test users and groups. I have django-auth-ldap configured mostly
>>>>>>>>> authenticating username/password against ldap instead of the internal
>>>>>>>>> database first. Once that is fully working the users will auto-populate
>>>>>>>>> into django and the groups should follow easily.
>>>>>>>>>
>>>>>>>>> Once that's done I'll be unblocked to finish the RBAC PoC. The
>>>>>>>>> rest of the parts are straightforward given the testing I've already done.
>>>>>>>>> More updates to come.
>>>>>>>>>
>>>>>>>>> On Mon, Jun 15, 2020 at 5:03 PM Brian Bouterse <
>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> I got the ldap reference implementation performing auth really
>>>>>>>>>> nicely against a test ldap with this guide:
>>>>>>>>>> https://www.nginx.com/blog/nginx-plus-authenticate-users/ Now
>>>>>>>>>> there are some new challenges though:
>>>>>>>>>>
>>>>>>>>>> * Great that we can auth users, but we need nginx to
>>>>>>>>>> extract-and-forward the group information to Pulp itself. That way a
>>>>>>>>>> middleware can create the user AND group info in the backend.
>>>>>>>>>> * we have to figure this out all again in Apache...
>>>>>>>>>>
>>>>>>>>>> Maybe we should be integrating Pulp directly against
>>>>>>>>>> django-auth-ldap [0]. I am going to try that next. The work I've done isn't
>>>>>>>>>> 100% reusable there, but most of it is because the test server and configs
>>>>>>>>>> I used can all be reused directly with django-auth-ldap. The concern with
>>>>>>>>>> this approach is that we would be supporting LDAP (and transitively Active
>>>>>>>>>> Directory) but are there other directory services Pulp needs to support?
>>>>>>>>>>
>>>>>>>>>> I also emailed Bin Li asking for info on how their user and group
>>>>>>>>>> management works.
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 9, 2020 at 11:48 AM Adrian Likins <alikins at redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 5, 2020 at 8:23 PM Brian Bouterse <
>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 1) django admin (the built in django UI) will be the mechanism
>>>>>>>>>>>> administrators use to assign permissions to users and groups. This means
>>>>>>>>>>>> the use of django admin with pulp is very likely (to me).
>>>>>>>>>>>>
>>>>>>>>>>>> Hopefully https://github.com/pulp/pulpcore/pull/705 will be
>>>>>>>>>>> useful here.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> 2) externally defined users and groups will need to be
>>>>>>>>>>>> "replicated" to django's db at login time, probably using headers from the
>>>>>>>>>>>> webserver This is consistent w/ the approach recommended here:
>>>>>>>>>>>> https://www.adelton.com/django/external-authentication-for-django-projects
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This is more or less what galaxy_ng ends up doing, at least for
>>>>>>>>>>> the scenarios where it runs hosted with external SSO.
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/auth/auth.py#L51 for
>>>>>>>>>>> example.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200701/e2ce36fb/attachment.htm>