[Pulp-dev] RBAC Status Thread

Fri Jul 10 06:01:26 UTC 2020

On Thu, Jul 9, 2020 at 11:20 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>
> This is all done. I've pushed my code to the links below. You can now use get_current_authenticated_user in both viewsets and tasks and it will give you the user, which lets you use signals to automatically add object-level permissions anywhere automatically. Also tasks have RBAC themselves (including cancelation and deletion restriction), and support queryset filtering.
>
> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC
> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>
> Here are a few random topics I've been thinking about:
>
> ## What should the default policy be?
>
> I propose Pulp ship with a very permissive policy, and this is effectively with RBAC off (also a requirement, e.g. Katello's usage). Each viewset would have an AccessPolicy but it would by default ship with the following single rule which allows everything.
>
> {
>     "action": ["*"],
>     "principal": "authenticated",
>     "effect": "allow",
> }

I think, we can satisfy Katello as well with a very restrictive
policy. They are using the one and only available account who happens
to be admin anyway.

>
> ## How can we adopt user-controllable policies?
>
> The highest goal here is to have users in total control of the policy. This is in-line with what drf-access-policy recommends:  https://rsinger86.github.io/drf-access-policy/loading_external_source/
>
> I propose we store the json policy for each Viewset, e.g. FileRemoteAccessPolicy which governs File Remotes as a blob that can by editable via the API itself. This allows us to ship a simple policy like ^ and allow users to do what they want. We can provide many example policies also.
>
> [idea] We could make a sub-resource called "access_policy" so for FileRemote for example you could read the policy here:
>
> GET /pulp/api/v3/remotes/file/file/access_policy
> PUT /pulp/api/v3/remotes/file/file/access_policy
>
> It would not support PATCH because we don't allow partial policy changes, it's one blob. It wouldn't support DELETE because there is always a policy. It wouldn't support POST because you're modifying a resource in-place.
>
> ## Requirements driving this
>
> In my discussions with galaxy_ng their deployment on the cloud versus on-premise will need a drastically different policy. This tells me that we need pluggable policies at initial launch.
>
> ## My main concerns at this point:
>
> 1) Galaxy_ng also needs flexibility between their cloud versus on-premise deployment regarding what the obj-level permissions created are. If ^ is implemented the user can have total control over the policy, but zero control over what permissions are assigned to newly created objects. For example that happens in this signal here:  https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC?expand=1#diff-bdb7fbb7c563d6651a309aff97001cb6R8-R12
>
> 2) Also can an admin "fix" object-level permissions? Right now they are only in the DB. I suspect the answer is for us to turn on more django-guardian admin integration:  https://github.com/django-guardian/django-guardian#admin-integration
>
> I have to think about this last part more... Meanwhile I plan to go forward with the user-controllable policies next. Input and feedback is welcome.
>
> Cheers,
> Brian
>
>
>
>
> On Wed, Jul 8, 2020 at 4:53 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>
>> My next goal is to have object-level permissions assigned through signals so that anywhere you save the model the permissions are correctly created. To do this I need to get a few things working:
>>
>> 1) Move the permissions creation to the signals [done]
>> 2) Have the user be well-known, this is accomplished by pulpcore picking up this dependency https://github.com/PaesslerAG/django-currentuser [done]
>> 3) Have the user information persisted in tasks so signals can work there also ... there are two obvious options
>>
>> option a) Have tasks themselves have RBAC which we know we need anyway, then query the RBAC permissions inside a task to determine which user has the object level permissions for this task. This is what I'm currently prototyping. It at least will provide a mechanism we can use for now until we find something better.
>>
>> option b) Add a user field to the Task itself and have the RBAC assign permissions to them. The concern with this is that while this is nice, it would be nicer if we had this type of visibility on everything in Pulp from an auditing project dedicated to keeping this type of info. Option (b) is still viable, just maybe not the best way. Comments/feedback is welcome.
>>
>> Also I've collaborated with @alikins from galaxy_ng project and together we wrote this requirements doc for their plugin:  https://hackmd.io/JisLkfyeT2myAD2khEwU0A
>>
>> On Wed, Jul 1, 2020 at 6:43 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>
>>> The demo advertisement for tomorrow is here:  https://www.redhat.com/archives/pulp-dev/2020-June/msg00076.html
>>>
>>> On Wed, Jul 1, 2020 at 6:41 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>
>>>> Another productive RBAC day! See the latest code at the links below. Here's what's new:
>>>>
>>>> * policy is now shorter thanks to machinery checking both model-level and object-level permissions with one call. The other two are also available
>>>> * sync is now restricted on both 'modify_repo_content' permissions AND read permission on the remote being used to sync
>>>> * modify is now restricted on 'modify_repo_content' permission
>>>> * moved the permission checking machinery to be "global checks"
>>>> * added data migration that sets is_staff=True, so the django-admin interface can be used (this is getting a slight rework tomorrow morning tho)
>>>>
>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC
>>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>>>
>>>> Tomorrow's demo is advertised here. It will also include an overview of some of the unsolved problems with some possible solutions.
>>>>
>>>> Cheers,
>>>> Brian
>>>>
>>>>
>>>> On Tue, Jun 30, 2020 at 5:08 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>
>>>>> Today I accomplished a few more things:
>>>>>
>>>>> * finished my ldap notes: https://hackmd.io/ED9UpscNSRW86Le3xNzVeg
>>>>> * moving the checks from a mixin to be "global checks" so they are available everywhere, this is a feature from drf-access-policy:  https://rsinger86.github.io/drf-access-policy/reusable_conditions/
>>>>> * added a has_obj_or_module_perms method allowing policy writers to just use that instad of carrying "two entries" in the policy, one for model-level, one for object-level
>>>>>
>>>>> Need to:
>>>>> * clean up the "sync" policy code
>>>>> * Add global condition check facilities for the perms of a 'remote' param
>>>>> * add policy language restricting the /modify/ endpoint also for FileRepository
>>>>> * push my code
>>>>>
>>>>> New Challenge: We need to also have the permissions assignments happen for objects created by tasks. django-guardian recommends this happen inside signals (https://django-guardian.readthedocs.io/en/latest/userguide/assign.html#assigning-permissions-inside-signals). The challenge (thanks @mdellweg for identifying) is that the user/group context information is well-known in the viewset but not in a task. Soooooo ... the idea is:
>>>>>
>>>>> 1. Switch the perms addition to the model itself via signals so it's automatic everywhere (including in tasks)
>>>>> 2. Preserve the user and group "request context" into the tasking system. I can see a straightforward path to how to do this so I plan to prototype this soon also.
>>>>>
>>>>> Feedback is welcome!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jun 26, 2020 at 6:16 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>
>>>>>> Today I got the "sync" RBAC working, but I need to give it some more thought. The extra challenge with this parts is that "having permission to read a Remote" is already defined in one place, on FileRemoteAccessPolicy, yet the AccessPolicy that needs to perform the enforcement is FileRepositoryAccessPolicy for its "sync" action. This is a bit challenging considering the following goals:
>>>>>>
>>>>>> * We don't want to duplicate code, e.g. having the FileRepositoryAccessControl begin to inspect permissions for FileRemote directly, when FileRemoteAccessPolicy already does that
>>>>>> * Currently permissions are granted at two levels: Model-level and File-level permissions and permissions are granted from either level.
>>>>>> * We want to keep the policy in charge. If we start to bury the behavior in methods and functions then policy writers are no longer in control.
>>>>>>
>>>>>> All of ^ together tells me that I should work on creating two things next:
>>>>>> 1) A way for policy writers to express which parameter refers to objects that also need their permissions checked. For example the policy should be able to say "remote is a parameter and it needs X permission". This is akin to the has_module_level_perms and has_obj_level_perms here except we also need to identify which parameter is being checked instead of the object the AccessPolicy itself governs.
>>>>>> 2) A single way to check model-level and object-level permissions at once and allow if *either* passes. We would still allow policy writers to call either model-level or file-level checks also.
>>>>>>
>>>>>> I'll work on ^ next. Ideas and feedback are welcome. I pushed no new code today because it's a mess and not runnable at my stopping point.
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 25, 2020 at 6:18 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>
>>>>>>> Here's another push to the branch (it includes the following additions):  https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>
>>>>>>> * A FileRepositoryAccessPolicy which provides RBAC for Repositories (not yet sync)
>>>>>>> * A new Mixin allowing the two policies to share some common components
>>>>>>>
>>>>>>> Next up:
>>>>>>> * have the pup_file define the fileContentAdmin group programmatically
>>>>>>> * Extend the FileRepositoryAccessPolicy to restrict sync operations
>>>>>>> * Write up and organize the PoC into a clear, organized format
>>>>>>>
>>>>>>> Also of interest today @ttereshc and I had a great convo asking what to do about potential problems when we use Django groups to be a "role". My write up will address this in more detail than I can go into here. We are also looking at what the django-role-permissions project could offer us:  https://django-role-permissions.readthedocs.io/en/stable/utils.html
>>>>>>>
>>>>>>> I expect the PoC to be done by tomorrow and write-up by Monday, so I'm going to schedule the public review meeting for next week towards the end of the week.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 24, 2020 at 5:49 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>>
>>>>>>>> Moar progress! Today the following things got done: Today's changes are available here:  https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>>
>>>>>>>> * Got scoped querysets working! This restricts list views to only show objects a user has permissions to view. A db reset was all that was needed I think I didn't have all the changes in when I applied my earlier migrations
>>>>>>>> * Added "detail view" restriction, and while it's in the policy and working DRF does a strange thing on "retrieve" where if it's not in the queryset (due to scoping ^) the user receives a 404, not a permission denied
>>>>>>>> * Got permissions cleaning up on resource deletion now too
>>>>>>>>
>>>>>>>> Next up:
>>>>>>>> * have the pup_file define the fileContentAdmin group programmatically
>>>>>>>> * Make similar policies for FileRepository which governs itself and the "sync" action
>>>>>>>> * Write up and organize the PoC into a clear, organized format
>>>>>>>>
>>>>>>>> Questions and feedback are welcome!
>>>>>>>>
>>>>>>>> On Tue, Jun 23, 2020 at 5:54 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>> Lots of progress today! I have a mostly-complete policy for RBAC for FileRemote. It's surprising how little code all of this ended up being.
>>>>>>>>>
>>>>>>>>> Here's the actual RBAC stuff, it's all in pulp_file:   https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>>> Here's the parts that go in core. Note the LDAP stuff is all optional, the only real requirement are two lines 1) enabling guardian in INSTALLED_APPS and 2) adding it as an AuthenticationBackend:  https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>>>>>>>>
>>>>>>>>> I have some "how to use notes" here:  https://hackmd.io/DRqGFyRsSDmN7E4TtOPf-w  The idea is that it implements the FileRemote portions of this requirements docs:   https://hackmd.io/kZ1oYp8TTkeuC5KL_ffjGQ
>>>>>>>>>
>>>>>>>>> Here is the short list of things for FileRemote that still don't work. This is mainly so I remember what to do next. :)
>>>>>>>>> * The get_objects_for_user from DjangoGuardian I don't think it likes Master/Detail or maybe it's how/where I'm using it. I haven't yet debugged this. For this reason it doesn't provide list restriction
>>>>>>>>> * It still needs "detail view" restriction. This is straightforward.
>>>>>>>>> * The group should be programmatically defined, in this case it was "defined in LDAP". It could *also* live in LDAP (or other external group definition system) but the plugin builds permissions off of it so it should also define it. This is easy.
>>>>>>>>>
>>>>>>>>> Feedback is welcome. I'm going to continue building this and then schedule a public review of FileRemote, Content modification for file repos, and sync restriction next week.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 22, 2020 at 5:14 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>> # ldap PoC updates
>>>>>>>>>> Now users, groups, and group membership are populating from ldap automatically on login (with auth backed by ldap also)! I'll be sharing my configs for both ldap and how to configure django-auth-ldap here soon in an organized way. This was done with django-auth-ldap and 0 customization to pulp code. It's 100% enabled through settings so this work is more of an approach we can document for users that they can enable and not a feature Pulp ships itself.
>>>>>>>>>>
>>>>>>>>>> # django-admin progress
>>>>>>>>>> Thanks to @alikins existing PRs, I got django admin enabled and able to view/edit users, groups, group membership, and permissions at both the user and group levels. This is important because this will be the primary mechanism of administrators. This part is looking good.
>>>>>>>>>>
>>>>>>>>>> # new resources to help us out
>>>>>>>>>> Through collaboration with @ttereshc and someone off list named @adelton (who actually authored this reference approach I referenced early on in this exploration), this very cool repository of testing tools was identified:  https://github.com/adelton/webauthinfra  It has a treasure trove of testing containers which Pulp devs in the future can test against. It keeps the user/group check in the apache which is fine alternative to the django-auth-ldap approach above. Pulp doesn't have to choose, it could work with either just configured differently. The pending PoC outline will go over these alternative approaches in detail.
>>>>>>>>>>
>>>>>>>>>> # Next Steps:  back to the PoC itself
>>>>>>>>>> Now that we have demonstrated good options of external users/groups/membership loading into Pulp we can confidently move back to finishing the RBAC PoC itself. I've started back into this. So the remaining work are the two steps below:
>>>>>>>>>>
>>>>>>>>>> 1. Finish the PoC that uses RBAC to restrict remotes, sync, and repository content modification. Currently I prototyped restriction of operations on Remotes, but I need to replicate the access policies to Repositories and Sync next.
>>>>>>>>>> 2. Write it up and share it.
>>>>>>>>>> 3. Schedule public meeting to review it (targeting next-week)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jun 19, 2020 at 5:09 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I got the LDAP users both authenticating and importing into Pulp! Next I'll do the groups and then I think the ldap parts will be done.
>>>>>>>>>>>
>>>>>>>>>>> FYI: I'm going to write up the implementation design and have that come with this proof of concept code . This will let us know what choices it makes, why it makes them, and we can determine if these are the right choices together.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 17, 2020 at 4:57 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I got a lot further on this today. I have the test ldap setup with several test users and groups. I have django-auth-ldap configured mostly authenticating username/password against ldap instead of the internal database first. Once that is fully working the users will auto-populate into django and the groups should follow easily.
>>>>>>>>>>>>
>>>>>>>>>>>> Once that's done I'll be unblocked to finish the RBAC PoC. The rest of the parts are straightforward given the testing I've already done. More updates to come.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jun 15, 2020 at 5:03 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I got the ldap reference implementation performing auth really nicely against a test ldap with this guide:  https://www.nginx.com/blog/nginx-plus-authenticate-users/ Now there are some new challenges though:
>>>>>>>>>>>>>
>>>>>>>>>>>>> * Great that we can auth users, but we need nginx to extract-and-forward the group information to Pulp itself. That way a middleware can create the user AND group info in the backend.
>>>>>>>>>>>>> * we have to figure this out all again in Apache...
>>>>>>>>>>>>>
>>>>>>>>>>>>> Maybe we should be integrating Pulp directly against django-auth-ldap [0]. I am going to try that next. The work I've done isn't 100% reusable there, but most of it is because the test server and configs I used can all be reused directly with django-auth-ldap. The concern with this approach is that we would be supporting LDAP (and transitively Active Directory) but are there other directory services Pulp needs to support?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I also emailed Bin Li asking for info on how their user and group management works.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jun 9, 2020 at 11:48 AM Adrian Likins <alikins at redhat.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jun 5, 2020 at 8:23 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) django admin (the built in django UI) will be the mechanism administrators use to assign permissions to users and groups. This means the use of django admin with pulp is very likely (to me).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hopefully https://github.com/pulp/pulpcore/pull/705 will be useful here.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) externally defined users and groups will need to be "replicated" to django's db at login time, probably using headers from the webserver This is consistent w/ the approach recommended here:  https://www.adelton.com/django/external-authentication-for-django-projects
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is more or less what galaxy_ng ends up doing, at least for the scenarios where it runs hosted with external SSO.
>>>>>>>>>>>>>> https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/auth/auth.py#L51 for example.
>>>>>>>>>>>>>>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev