[Pulp-dev] RBAC Status Thread

[Brian Bouterse]

This is all done. I've pushed my code to the links below. You can now use
get_current_authenticated_user in both viewsets and tasks and it will give
you the user, which lets you use signals to automatically add object-level
permissions anywhere automatically. Also tasks have RBAC themselves
(including cancelation and deletion restriction), and support queryset
filtering.

https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC
https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC

Here are a few random topics I've been thinking about:

## What should the default policy be?

I propose Pulp ship with a very permissive policy, and this is effectively
with RBAC off (also a requirement, e.g. Katello's usage). Each viewset
would have an AccessPolicy but it would by default ship with the following
single rule which allows everything.

{
    "action": ["*"],
    "principal": "authenticated",
    "effect": "allow",
}

## How can we adopt user-controllable policies?

The highest goal here is to have users in total control of the policy. This
is in-line with what drf-access-policy recommends:
https://rsinger86.github.io/drf-access-policy/loading_external_source/

I propose we store the json policy for each Viewset, e.g.
FileRemoteAccessPolicy which governs File Remotes as a blob that can by
editable via the API itself. This allows us to ship a simple policy like ^
and allow users to do what they want. We can provide many example policies
also.

[idea] We could make a sub-resource called "access_policy" so for
FileRemote for example you could read the policy here:

GET /pulp/api/v3/remotes/file/file/access_policy
PUT /pulp/api/v3/remotes/file/file/access_policy

It would not support PATCH because we don't allow partial policy changes,
it's one blob. It wouldn't support DELETE because there is always a policy.
It wouldn't support POST because you're modifying a resource in-place.

## Requirements driving this

In my discussions with galaxy_ng their deployment on the cloud versus
on-premise will need a drastically different policy. This tells me that we
need pluggable policies at initial launch.

## My main concerns at this point:

1) Galaxy_ng also needs flexibility between their cloud versus on-premise
deployment regarding what the obj-level permissions created are. If ^ is
implemented the user can have total control over the policy, but zero
control over what permissions are assigned to newly created objects. For
example that happens in this signal here:
https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC?expand=1#diff-bdb7fbb7c563d6651a309aff97001cb6R8-R12

2) Also can an admin "fix" object-level permissions? Right now they are
only in the DB. I suspect the answer is for us to turn on more
django-guardian admin integration:
https://github.com/django-guardian/django-guardian#admin-integration

I have to think about this last part more... Meanwhile I plan to go forward
with the user-controllable policies next. Input and feedback is welcome.

Cheers,
Brian




On Wed, Jul 8, 2020 at 4:53 PM Brian Bouterse <bmbouter at redhat.com> wrote:

> My next goal is to have object-level permissions assigned through signals
> so that anywhere you save the model the permissions are correctly created.
> To do this I need to get a few things working:
>
> 1) Move the permissions creation to the signals [done]
> 2) Have the user be well-known, this is accomplished by pulpcore picking
> up this dependency https://github.com/PaesslerAG/django-currentuser [done]
> 3) Have the user information persisted in tasks so signals can work there
> also ... there are two obvious options
>
> option a) Have tasks themselves have RBAC which we know we need anyway,
> then query the RBAC permissions inside a task to determine which user has
> the object level permissions for this task. This is what I'm currently
> prototyping. It at least will provide a mechanism we can use for now until
> we find something better.
>
> option b) Add a user field to the Task itself and have the RBAC assign
> permissions to them. The concern with this is that while this is nice, it
> would be nicer if we had this type of visibility on everything in Pulp from
> an auditing project dedicated to keeping this type of info. Option (b) is
> still viable, just maybe not the best way. Comments/feedback is welcome.
>
> Also I've collaborated with @alikins from galaxy_ng project and together
> we wrote this requirements doc for their plugin:
> https://hackmd.io/JisLkfyeT2myAD2khEwU0A
>
> On Wed, Jul 1, 2020 at 6:43 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>
>> The demo advertisement for tomorrow is here:
>> https://www.redhat.com/archives/pulp-dev/2020-June/msg00076.html
>>
>> On Wed, Jul 1, 2020 at 6:41 PM Brian Bouterse <bmbouter at redhat.com>
>> wrote:
>>
>>> Another productive RBAC day! See the latest code at the links below.
>>> Here's what's new:
>>>
>>> * policy is now shorter thanks to machinery checking both model-level
>>> and object-level permissions with one call. The other two are also available
>>> * sync is now restricted on both 'modify_repo_content' permissions AND
>>> read permission on the remote being used to sync
>>> * modify is now restricted on 'modify_repo_content' permission
>>> * moved the permission checking machinery to be "global checks"
>>> * added data migration that sets is_staff=True, so the django-admin
>>> interface can be used (this is getting a slight rework tomorrow morning tho)
>>>
>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC
>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>>
>>> Tomorrow's demo is advertised here. It will also include an overview of
>>> some of the unsolved problems with some possible solutions.
>>>
>>> Cheers,
>>> Brian
>>>
>>>
>>> On Tue, Jun 30, 2020 at 5:08 PM Brian Bouterse <bmbouter at redhat.com>
>>> wrote:
>>>
>>>> Today I accomplished a few more things:
>>>>
>>>> * finished my ldap notes: https://hackmd.io/ED9UpscNSRW86Le3xNzVeg
>>>> * moving the checks from a mixin to be "global checks" so they are
>>>> available everywhere, this is a feature from drf-access-policy:
>>>> https://rsinger86.github.io/drf-access-policy/reusable_conditions/
>>>> * added a has_obj_or_module_perms method allowing policy writers to
>>>> just use that instad of carrying "two entries" in the policy, one for
>>>> model-level, one for object-level
>>>>
>>>> Need to:
>>>> * clean up the "sync" policy code
>>>> * Add global condition check facilities for the perms of a 'remote'
>>>> param
>>>> * add policy language restricting the /modify/ endpoint also for
>>>> FileRepository
>>>> * push my code
>>>>
>>>> New Challenge: We need to also have the permissions assignments happen
>>>> for objects created by tasks. django-guardian recommends this happen inside
>>>> signals (
>>>> https://django-guardian.readthedocs.io/en/latest/userguide/assign.html#assigning-permissions-inside-signals).
>>>> The challenge (thanks @mdellweg for identifying) is that the user/group
>>>> context information is well-known in the viewset but not in a task. Soooooo
>>>> ... the idea is:
>>>>
>>>> 1. Switch the perms addition to the model itself via signals so it's
>>>> automatic everywhere (including in tasks)
>>>> 2. Preserve the user and group "request context" into the tasking
>>>> system. I can see a straightforward path to how to do this so I plan to
>>>> prototype this soon also.
>>>>
>>>> Feedback is welcome!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Jun 26, 2020 at 6:16 PM Brian Bouterse <bmbouter at redhat.com>
>>>> wrote:
>>>>
>>>>> Today I got the "sync" RBAC working, but I need to give it some more
>>>>> thought. The extra challenge with this parts is that "having permission to
>>>>> read a Remote" is already defined in one place, on FileRemoteAccessPolicy,
>>>>> yet the AccessPolicy that needs to perform the enforcement is
>>>>> FileRepositoryAccessPolicy for its "sync" action. This is a bit challenging
>>>>> considering the following goals:
>>>>>
>>>>> * We don't want to duplicate code, e.g. having the
>>>>> FileRepositoryAccessControl begin to inspect permissions for FileRemote
>>>>> directly, when FileRemoteAccessPolicy already does that
>>>>> * Currently permissions are granted at two levels: Model-level and
>>>>> File-level permissions and permissions are granted from either level.
>>>>> * We want to keep the policy in charge. If we start to bury the
>>>>> behavior in methods and functions then policy writers are no longer in
>>>>> control.
>>>>>
>>>>> All of ^ together tells me that I should work on creating two things
>>>>> next:
>>>>> 1) A way for policy writers to express which parameter refers to
>>>>> objects that also need their permissions checked. For example the policy
>>>>> should be able to say "remote is a parameter and it needs X permission".
>>>>> This is akin to the has_module_level_perms and has_obj_level_perms here
>>>>> except we also need to identify which parameter is being checked instead of
>>>>> the object the AccessPolicy itself governs.
>>>>> 2) A single way to check model-level and object-level permissions at
>>>>> once and allow if *either* passes. We would still allow policy writers to
>>>>> call either model-level or file-level checks also.
>>>>>
>>>>> I'll work on ^ next. Ideas and feedback are welcome. I pushed no new
>>>>> code today because it's a mess and not runnable at my stopping point.
>>>>>
>>>>>
>>>>> On Thu, Jun 25, 2020 at 6:18 PM Brian Bouterse <bmbouter at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Here's another push to the branch (it includes the following
>>>>>> additions):
>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>
>>>>>> * A FileRepositoryAccessPolicy which provides RBAC for Repositories
>>>>>> (not yet sync)
>>>>>> * A new Mixin allowing the two policies to share some common
>>>>>> components
>>>>>>
>>>>>> Next up:
>>>>>> * have the pup_file define the fileContentAdmin group programmatically
>>>>>> * Extend the FileRepositoryAccessPolicy to restrict sync operations
>>>>>> * Write up and organize the PoC into a clear, organized format
>>>>>>
>>>>>> Also of interest today @ttereshc and I had a great convo asking what
>>>>>> to do about potential problems when we use Django groups to be a "role". My
>>>>>> write up will address this in more detail than I can go into here. We are
>>>>>> also looking at what the django-role-permissions project could offer us:
>>>>>> https://django-role-permissions.readthedocs.io/en/stable/utils.html
>>>>>>
>>>>>> I expect the PoC to be done by tomorrow and write-up by Monday, so
>>>>>> I'm going to schedule the public review meeting for next week towards the
>>>>>> end of the week.
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 24, 2020 at 5:49 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Moar progress! Today the following things got done: Today's changes
>>>>>>> are available here:
>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>
>>>>>>> * Got scoped querysets working! This restricts list views to only
>>>>>>> show objects a user has permissions to view. A db reset was all that was
>>>>>>> needed I think I didn't have all the changes in when I applied my earlier
>>>>>>> migrations
>>>>>>> * Added "detail view" restriction, and while it's in the policy and
>>>>>>> working DRF does a strange thing on "retrieve" where if it's not in the
>>>>>>> queryset (due to scoping ^) the user receives a 404, not a permission denied
>>>>>>> * Got permissions cleaning up on resource deletion now too
>>>>>>>
>>>>>>> Next up:
>>>>>>> * have the pup_file define the fileContentAdmin group
>>>>>>> programmatically
>>>>>>> * Make similar policies for FileRepository which governs itself and
>>>>>>> the "sync" action
>>>>>>> * Write up and organize the PoC into a clear, organized format
>>>>>>>
>>>>>>> Questions and feedback are welcome!
>>>>>>>
>>>>>>> On Tue, Jun 23, 2020 at 5:54 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Lots of progress today! I have a mostly-complete policy for RBAC
>>>>>>>> for FileRemote. It's surprising how little code all of this ended up being.
>>>>>>>>
>>>>>>>> Here's the actual RBAC stuff, it's all in pulp_file:
>>>>>>>> https://github.com/pulp/pulp_file/compare/master...bmbouter:rbac-PoC?expand=1
>>>>>>>> Here's the parts that go in core. Note the LDAP stuff is all
>>>>>>>> optional, the only real requirement are two lines 1) enabling guardian in
>>>>>>>> INSTALLED_APPS and 2) adding it as an AuthenticationBackend:
>>>>>>>> https://github.com/pulp/pulpcore/compare/master...bmbouter:rbac-PoC
>>>>>>>>
>>>>>>>> I have some "how to use notes" here:
>>>>>>>> https://hackmd.io/DRqGFyRsSDmN7E4TtOPf-w  The idea is that it
>>>>>>>> implements the FileRemote portions of this requirements docs:
>>>>>>>> https://hackmd.io/kZ1oYp8TTkeuC5KL_ffjGQ
>>>>>>>>
>>>>>>>> Here is the short list of things for FileRemote that still don't
>>>>>>>> work. This is mainly so I remember what to do next. :)
>>>>>>>> * The get_objects_for_user
>>>>>>>> <https://django-guardian.readthedocs.io/en/latest/api/guardian.shortcuts.html#get-objects-for-user>
>>>>>>>> from DjangoGuardian I don't think it likes Master/Detail or maybe it's
>>>>>>>> how/where I'm using it. I haven't yet debugged this. For this reason it
>>>>>>>> doesn't provide list restriction
>>>>>>>> * It still needs "detail view" restriction. This is straightforward.
>>>>>>>> * The group should be programmatically defined, in this case it was
>>>>>>>> "defined in LDAP". It could *also* live in LDAP (or other external group
>>>>>>>> definition system) but the plugin builds permissions off of it so it should
>>>>>>>> also define it. This is easy.
>>>>>>>>
>>>>>>>> Feedback is welcome. I'm going to continue building this and then
>>>>>>>> schedule a public review of FileRemote, Content modification for file
>>>>>>>> repos, and sync restriction next week.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 22, 2020 at 5:14 PM Brian Bouterse <bmbouter at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> # ldap PoC updates
>>>>>>>>> Now users, groups, and group membership are populating from ldap
>>>>>>>>> automatically on login (with auth backed by ldap also)! I'll be sharing my
>>>>>>>>> configs for both ldap and how to configure django-auth-ldap
>>>>>>>>> <https://django-auth-ldap.readthedocs.io/en/latest/example.html>
>>>>>>>>> here soon in an organized way. This was done with django-auth-ldap and 0
>>>>>>>>> customization to pulp code. It's 100% enabled through settings so this work
>>>>>>>>> is more of an approach we can document for users that they can enable and
>>>>>>>>> not a feature Pulp ships itself.
>>>>>>>>>
>>>>>>>>> # django-admin progress
>>>>>>>>> Thanks to @alikins existing PRs, I got django admin enabled and
>>>>>>>>> able to view/edit users, groups, group membership, and permissions at both
>>>>>>>>> the user and group levels. This is important because this will be the
>>>>>>>>> primary mechanism of administrators. This part is looking good.
>>>>>>>>>
>>>>>>>>> # new resources to help us out
>>>>>>>>> Through collaboration with @ttereshc and someone off list named
>>>>>>>>> @adelton (who actually authored this reference approach
>>>>>>>>> <https://www.adelton.com/django/external-authentication-for-django-projects>
>>>>>>>>> I referenced early on in this exploration), this very cool repository of
>>>>>>>>> testing tools was identified:
>>>>>>>>> https://github.com/adelton/webauthinfra  It has a treasure trove
>>>>>>>>> of testing containers which Pulp devs in the future can test against. It
>>>>>>>>> keeps the user/group check in the apache which is fine alternative to the
>>>>>>>>> django-auth-ldap approach above. Pulp doesn't have to choose, it could work
>>>>>>>>> with either just configured differently. The pending PoC outline will go
>>>>>>>>> over these alternative approaches in detail.
>>>>>>>>>
>>>>>>>>> # Next Steps:  back to the PoC itself
>>>>>>>>> Now that we have demonstrated good options of external
>>>>>>>>> users/groups/membership loading into Pulp we can confidently move back to
>>>>>>>>> finishing the RBAC PoC itself. I've started back into this. So the
>>>>>>>>> remaining work are the two steps below:
>>>>>>>>>
>>>>>>>>> 1. Finish the PoC that uses RBAC to restrict remotes, sync, and
>>>>>>>>> repository content modification. Currently I prototyped restriction of
>>>>>>>>> operations on Remotes, but I need to replicate the access policies to
>>>>>>>>> Repositories and Sync next.
>>>>>>>>> 2. Write it up and share it.
>>>>>>>>> 3. Schedule public meeting to review it (targeting next-week)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jun 19, 2020 at 5:09 PM Brian Bouterse <
>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> I got the LDAP users both authenticating and importing into Pulp!
>>>>>>>>>> Next I'll do the groups and then I think the ldap parts will be done.
>>>>>>>>>>
>>>>>>>>>> FYI: I'm going to write up the implementation design and have
>>>>>>>>>> that come with this proof of concept code . This will let us know what
>>>>>>>>>> choices it makes, why it makes them, and we can determine if these are the
>>>>>>>>>> right choices together.
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 17, 2020 at 4:57 PM Brian Bouterse <
>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I got a lot further on this today. I have the test ldap setup
>>>>>>>>>>> with several test users and groups. I have django-auth-ldap configured
>>>>>>>>>>> mostly authenticating username/password against ldap instead of the
>>>>>>>>>>> internal database first. Once that is fully working the users will
>>>>>>>>>>> auto-populate into django and the groups should follow easily.
>>>>>>>>>>>
>>>>>>>>>>> Once that's done I'll be unblocked to finish the RBAC PoC. The
>>>>>>>>>>> rest of the parts are straightforward given the testing I've already done.
>>>>>>>>>>> More updates to come.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 15, 2020 at 5:03 PM Brian Bouterse <
>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I got the ldap reference implementation performing auth really
>>>>>>>>>>>> nicely against a test ldap with this guide:
>>>>>>>>>>>> https://www.nginx.com/blog/nginx-plus-authenticate-users/ Now
>>>>>>>>>>>> there are some new challenges though:
>>>>>>>>>>>>
>>>>>>>>>>>> * Great that we can auth users, but we need nginx to
>>>>>>>>>>>> extract-and-forward the group information to Pulp itself. That way a
>>>>>>>>>>>> middleware can create the user AND group info in the backend.
>>>>>>>>>>>> * we have to figure this out all again in Apache...
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe we should be integrating Pulp directly against
>>>>>>>>>>>> django-auth-ldap [0]. I am going to try that next. The work I've done isn't
>>>>>>>>>>>> 100% reusable there, but most of it is because the test server and configs
>>>>>>>>>>>> I used can all be reused directly with django-auth-ldap. The concern with
>>>>>>>>>>>> this approach is that we would be supporting LDAP (and transitively Active
>>>>>>>>>>>> Directory) but are there other directory services Pulp needs to support?
>>>>>>>>>>>>
>>>>>>>>>>>> I also emailed Bin Li asking for info on how their user and
>>>>>>>>>>>> group management works.
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jun 9, 2020 at 11:48 AM Adrian Likins <
>>>>>>>>>>>> alikins at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 5, 2020 at 8:23 PM Brian Bouterse <
>>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) django admin (the built in django UI) will be the
>>>>>>>>>>>>>> mechanism administrators use to assign permissions to users and groups.
>>>>>>>>>>>>>> This means the use of django admin with pulp is very likely (to me).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hopefully https://github.com/pulp/pulpcore/pull/705 will be
>>>>>>>>>>>>> useful here.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) externally defined users and groups will need to be
>>>>>>>>>>>>>> "replicated" to django's db at login time, probably using headers from the
>>>>>>>>>>>>>> webserver This is consistent w/ the approach recommended here:
>>>>>>>>>>>>>> https://www.adelton.com/django/external-authentication-for-django-projects
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is more or less what galaxy_ng ends up doing, at least
>>>>>>>>>>>>> for the scenarios where it runs hosted with external SSO.
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/auth/auth.py#L51 for
>>>>>>>>>>>>> example.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200709/114dcafa/attachment.htm>