[Freeipa-devel] [RFC] Migrating existing environments to Trust

Dmitri Pal dpal at redhat.com
Fri Apr 18 03:58:59 UTC 2014

On 04/17/2014 05:15 AM, Sumit Bose wrote:
> On Wed, Apr 16, 2014 at 09:02:00PM -0400, Dmitri Pal wrote:
>> On 04/15/2014 05:13 AM, Sumit Bose wrote:
>>> Hi,
>>> I have started to write a design page for 'Migrating existing
>>> environments to Trust'
>>> http://www.freeipa.org/page/V3/Migrating_existing_environments_to_Trust
>>> It shall cover https://fedorahosted.org/freeipa/ticket/3318 and
>>> https://fedorahosted.org/freeipa/ticket/3979 .
>>> I came across several questions which need answers so that more details
>>> can be added to the design. Besides that comments and suggestions are
>>> welcome as well.
>>> For your convenience I added the test below as well.
>>> bye,
>>> Sumit
>>> = Overview =
>>> This page illustrates how existing solutions which make AD users
>>> available to Linux hosts can be migrated to FreeIPA with Trusts. This
>>> includes migrations from the FreeIPA WinSync feature or environments
>>> where the AD users where correlated to NIS users.
>>> The common problem here is that some if not all attributes needed by a
>>> POSIX user or group must be overwritten or supplied by the IPA server.
>>> The link to the related AD object is preferably the SID but if it is not
>>> available on both sides the name of the object must be used. AD will
>>> keep the responsibility for authentication and provide the AD
>>> group-memberships of the object.
>>> = Use Cases =
>>> * Migration from the FreeIPA Sync solution to the FreeIPA Trust solution
>>> ** [https://fedorahosted.org/freeipa/ticket/3318 https://fedorahosted.org/freeipa/ticket/3318]
>>> * Migration/Consolidation of domains currently managed by other solutions, e.g. NIS
>>> ** [https://fedorahosted.org/freeipa/ticket/3979 https://fedorahosted.org/freeipa/ticket/3979] (contains some ideas about a solution)
>> As I mentioned in the ticket not only that. Based on conversations
>> with different potential consumers of the trust functionality the
>> ability to use existing POSIX attributes and manage them in IPA
>> while user accounts come from AD is a crucial next step.
> Thank you for your feedback, it was very helpful but I'm afraid it might
> also caused some new questions.
>>> = Design =
>>> In Ticket [https://fedorahosted.org/freeipa/ticket/3979 https://fedorahosted.org/freeipa/ticket/3979] two aspects of a design are already explained.
>>> # Instead of just offering a single override option the introduction of
>>>    views are suggested. A suitable client can ask for a specific view
>>>    with override options different from the default override
>>>    view.
>> Yes.
>>> Questions:
>>> #* Will the default view always be applied? I think it makes sense to
>>>     always apply it to get a consistent default behavior. If there is a
>>>     reason for a client to get the unmodified data a special view called
>>>     e.g. NO_OVERRIDE can be used. This is e.g. needed for the extdom plugin
>>>     to be able to send the raw data to the IPA client so that SSSD can use
>>>     different views on the client which might be needed for docker/container
>>>     use cases.
>> Sounds reasonable to have the default view and apply it always. If
>> the view does not contain posix attributes for the specific user we
>> should use dynamic mapping based on SIDs.
> Quite some time ago we have decided to not mix algorithmic mapping and
> manually managed POSIX IDs. E.g. think about the case where a view with
> POSIX ID was added for an existing user which already has an algorithmic
> ID assigned on some clients.

I think this is unfortunate but I can live with this though it is not best.
If there is a workaround to have two trusts or two ranges associated 
with the forest with different mapping properties I am fine but we need 
to have that be clearly explain on a page because people were asking 
about this and I unfortunately gave them the wrong answer becuase I was 
not aware of "the decision". IMO the decision seems contour intuitive to 
me. Here is the use cases that people explained to me:
"I had to put and manage my posix attributes in AD using third party 
solution. But as soon as I migrate to IPA I want the AD side of the 
company to stop managing posix attributes."
I was under wrong assumption that though we do not have views now they 
can do what we ask by allowing the algorithmic mapping to kick in for 
new users.
Frankly I do not see a reason why we can't assign posix attributes using 
algorithmic method if the posix attributes are not explicitly set. I can 
argue that this can be an optional fall back but it should be possible.
I expect that there will be bugs filed about it as people try. We will see.

> I think the admin has to decide what he wants.

He wants to use what he already has from AD but then use other mappings 
on top. Since we do not provide views yet the only expected option is to 
fall back to algorithmic mapping.

> Below you mentioned a
> migration use case where old users should keep their IDs but new users
> will get algorithmically generated ones. I think this is bad practice
> and technically is it next to impossible without additional admin
> effort to decide if a given AD user is an old or a new user.

May be but IMO it is a deployment choice.
I do not see a reason why we would not assign posix attributes always.
Even if they are not needed or used.
This way we can always get them if user suddenly logs into an operating 
system rather than logging just over the web to the applications.
In the current implementation the user would not be able to do it 
because before he can do it he has to contact admin and ask him to do 
something because he can't log into this VM that was setup for him.
I am not sure it is a ban practice. It is up to admin to decide.

By new users I mean users created after the migration point. It might be 
just a switch in a policy indicating that migration is complete and that 
would indicate that any users coming from AD without posix should be 
treated as new because for those ones that admins wanted the already set 
posix attributes in AD explicitly.

>   The admin
> either has to add special flags/attributes to the old AD user objects or
> we have to keep an immutable list of old users in IPA. Please note that
> this has to be done for groups as well. Imo it would be
> easier and safer for the admin to either do a full migration to
> algorithmically mapped IDs or manage all POSIX IDs manually on the IPA
> side.

I think it is not possible in real life.
They can migrate people they care about and set posix for those but not 
for everyone else but now with trust you will have a fall back 
algorythmic option if any of the AD users has to access Linuxz operating 

> Additionally I think that in your use case there might be even the need
> to manually manage POSIX IDs even for new user. E.g. in the case where a
> larger amount of new users is added to AD which where managed by a
> completely independent system before.

In this case they will add new users to AD and either set posix there or 
will use views. If they do not care about posix for merged users they 
will just not set anything letting the default algorithmic resolution to 
do the trick. If they do not like it then can not turn it on and be 
required to set UIDs manually.
>>> #* Will views only be applied to users from other domains or to IPA
>>>     users as well?
>> Design goal will be to allow them to be applied to all users.
> Why, what is the use case to override attribute of IPA users which
> cannot be solved by adding other attributes with the needed values to
> the IPA user object directly?

It seems that I was not clear with the use case.
There are two use cases really:

a) I have scattered NIS and LDAP domain, I would eventually clean them 
up but right now I want to migrate to IdM and not use 3rd party 
solution. Can IPA support zones like the 3rd party solution does?
Right now no.
As you see there is no mention of AD here.
b) I want to do a) but I also want to use AD as authoritative source so 
I want to consolidate everything in IPA but use trusts.
The simplified variant of the use case b) is: I have a single NIS or 
LDAP that I want to migrate to IPA from. In the past I was using ugly 
sync solutions from AD to NIS/LDAP that I want to replace with a trust. 
This means that I want to manage POSIX in IPA at least for the users 
that are migrated and I want to preserve what I had. For other users 
that are not migrated I will need a choice of managing posix in IPA 
myself or falling back to algorithmic if I do not want to set up POSIX 
myself any more. IPA creates new posix attributes for new IPA users it 
should act similarly for ad users without posix (if I told it to do so).

So if we generalize we see that view is really a way to assign different 
sets of posix attributes and primarily UID/GID to the same account and 
expose those to separate clients. May be it is all about compat views or 
CoS. This is implementation detail and for you to design.

>> Implementation goal will be to apply them to external users first.
>> Or I should say we need to figure out the procedure:
>> 1) Pre migration: AD with no POSIX, LDAP/NIS with POSIX for some/same users.
>> 2) Past migration: AD with no POSIX, no duplicate accounts in IdM,
>> POSIX attributes for AD users are migrated into a view.
>> How we do it is the question but we need to have a path. May be it
>> would require some kind of tool:
>> ipa create-view --check
>> This will go through all users in IPA and check them against
>> available trusted domains and would report which users will be
>> removed becuase they are found in a specific domain.
> I think it should only report conflicts. The admin should decide how to
> solve the conflict.
>> ipa create-view --create --except=<group list>
>> This will actually do the work but if there are special users that
>> need to be processed in some special way admin might temporarily put
>> them into a specific group and then exclude this group or groups
>> from cleanup.
>> Just a thought...
>>> #* Do we want stackable views?
>> We want client oriented views. I.e. views should be attached to host groups.
>> For the legacy clients there should be a way to expose a view as a
>> separate base=DN so that one set of legacy clients can be pointed to
>> one DN and another to another.
> I'm not sure if this will stretch the compat tree too much, I let
> Alexander and Nalin decided.

Keep in mind that we can potentially have different trees and point 
different clients to different trees like we do with automount locations.

for example
ipa-client-install ... --zone=zone3
will point to the cn=zone3, cn=zones, cn=accounts,...
instead of the main tree.

>> If the host based views and DN based views do not have data for a
>> specific user the data should be fetched from a global default view.
>> I think we can use CoS for that in conjunction with the compat
>> plugin but I would leave this to experts to decide.
>> Bottom line: one global fallback view and then specific views for
>> host groups and for legacy clients.
>> I know I might be asking a lot ;-)
> How shall this 'global default view' work? E.g. if there is an AD user
> without a POSIX ID in AD and algorithmic mapping is not enabled for this
> user.

How dows it work now? Based on what you said I assume user would fail to 
login because posix attributes are not set and I actually want him t 
succeeded by default. If I misunderstood you please explain.

>   If there is no override object for this user in the current view
> or if the override object does not contains a POSIX ID attribute where
> should the ID come from?

be algorithmic of allowed or admin has to set it manually if he does not 
want to fall back to algorithmic.

> Please note that it is a valid use-case that a user does not has a POSIX
> ID e.g. web applications. I think it is not necessary to forcefully try
> to assign some POSIX ID to a user.

Why not? What is the harm?

> If the admin decides that the user
> does not need one, why shall we try to create one?

Why would he not want to? What is the harm? Why he explicitly would care 
for us to not set the UID?

>>> #* Do we want to override everything or shall there be immutable
>>>     attributes like e.g. the SID or the user name?
>> I do not know why would overwrite the SID but I think that
>> overwriting the user name would give us an interesting aliasing
>> capability as a feature as a side effect.
> But is there a use case for such a feature which is not solved better by
> creating a new user with the given name? Please note the side effects,
> e.g. with respect to the home-directory path. Typically the user name is
> part of the of path. What if the original object has a home-directory
> path shall we override it implicitly as well or keep it and who shall
> decided? Or what about Kerberos principals, who shall user 'abc' know
> that his name was overridden and his Kerberos principal is
> 'xyz at AD.REALM'?

I am not sure there is a use case so frankly I would start with just 
overwriting UIDs and GIDs in the views and see if we really need to 
extend it to any other posix attribute and why. But ability to potential 
overwrite user name sounds interesting though not sure how useful it is 
and how applicable. We will see.

>>> #* Shall we allow different UIDs/GIDs in different views?
>> Yes.
> I hope the admin knows what he does in this case. I think it's similar
> like with the user name, is there really a user-case for this with
> cannot be solved better by creating a new user with the given UID? Think
> about what happens if a host is moved to a new host group e.g. to change
> the HBAC rules but by chance has now a different view with different
> UIDs?
> >From the OS point of view a user with a different UID is a different
> user, from the peoples point of view a user with a different name is a
> different user. I think what we achieve with allowing different values
> for user names and/or UIDs is to allow different users share the same
> password, Kerberos credentials etc and I'm not sure we really want this.

This is need for historical reasons. I would not repeat Simo's answer it 
is pretty all inclusive.

>>> #* I think overriding UIDs/GIDs should only be allowed for
>>>     ipa-ad-trust-posix idranges, mixing override with algorithmic mapping
>>>     does not make sense imo.
>> I think it does at least for the migration time. But it might not be
>> achievable.
>> The idea is that you really should be required to manage UID/GID for
>> the users manually via views if it is an old user.
>> If it is a new user that never was on the Linux side before the
>> algorithmic mapping might be preferred.
> see above
>> But I also think that this should be controlled by a policy and
>> admin would have to decide whether the IPA should generate UIDs or
>> he would prefer to set the manually explicitly.
> yes, this can already be controlled by the idrange type. But you have to
> choose either algorithmic or manual mapping you cannot have both in a
> given domain. What you can do is to create a domain in the AD forest for
> the old users and one for the new users. Now you can use manual mapping
> for the old-users-domain and algorithmic mapping for the
> new-users-domain.

If this requires moving users between domains on AD side then this is a 
non starter.
The more I read it the more I think that decision is wrong and it is bug.

>>> # The views will be stored in containers below
>>>    cn=views,cn=accounts,$SUFFIX with containers for users and groups. The
>>>    objectclasses will look similar to posixAccount and posixGroup
>>>    objectclasses but with only optional (MAY) attributes. Questions:
>>> #* Do we want to consider to allow to add arbitrary attributes to a view
>>>     to cover requests like "We have this beautiful application which can get
>>>     all user data via the SSSD D-BUS responder but our AD admin will not
>>>     extend the AD LDAP schema to add the required attributes. Can IPA add
>>>     them for users from trusted domains?"
>> Yes but probably not as phase 1. So it is a separate enhancement.
>>> #* It was suggested to use a UUID to reference the original objects. For
>>>     AD users and groups the SID would be a good choice because it is unique
>>>     and already contains a reference to the original domain. I wonder if we
>>>     should add  a type prefix like 'SID:' to be able to add other types like
>>>     'IPAUUID:domref:ef2b7400-a3c4-11e3-82e7-525400de2951' where domref will
>>>     be a reference to the other IPA domain. On the other hand a type can be
>>>     added later and if the type is missing a SID is assumed.
>>> On the SSSD side the views can be stored below the corresponding user or
>>> group object in the cache and the SYSDB API has to be enhanced to return
>>> merged results. The merging only happens in the responders (NSS, D-BUS)
>>> before sending data to the clients.
>> Why SSSD should know about different views?
>> It should know raw and not raw but it always sees one merged view
>> from the server.
>> What am I missing?
> You miss the docker/container use case where a central SSSD instance is
> used by instances with different host names which might belong to
> different host groups and hence require different views. So SSSD does
> not only need to know all views but the host groups (we already have
> them for HBAC) and their corresponding views as well.

True but i think that there SSSD would have to be configured in two 
different ways. For a simple case it can just be pointed to the zone view.
In the case of the docker it might have to be pointed to the main tree 
and then read all the overlays and create the views on the fly for the 
clients on the system.
But I would not go there yet. May be we would have SSSD in a service 
container and it would be easy to deploy another SSSD pointing to 
another view instead of re-implementing all the complexity of merging 
data to create views in SSSD again.

> Although this use case is work-in-progress I think we should include it
> from the start because the extra work to read all views quite small and
> might be a larger effort when added later.

We need to keep it minds but IMO multiple SSSDs in different containers 
seems a better solution to me for this use case and much easier achievable.
> bye,
> Sumit
>>> To manage the views a new set CLI tools/Web UI pages should be added.
>>> Depending if we would like to allow to override IPA users as well or
>>> only users from trusted domains the tools might get their own name
>>> space, ipa override-*, or can be added below the trust name space, ipa
>>> trust-override-*.  It has to be noted that changes of a view will only
>>> be visible on the client after the related cached object is expired.
>> OK.
>>> = Implementation =
>> See comments above. I hope they give enough hints for
>> implementation. At least for the first pass.
>>> =Feature Management =
>>> == UI ==
>>> == CLI ==
>>> = Major configuration options and enablement =
>>> Any configuration options? Any commands to enable/disable the feature or
>>> turn on/off its parts?
>>> = Replication =
>>> Any impact on replication?
>>> = Updates and Upgrades =
>>> Any impact on updates and upgrades?
>>> = Dependencies =
>>> Any new package and library dependencies.
>>> = External Impact =
>>> Impact on other development teams and components
>>> = Backup and Restore =
>>> Any files or configuration that needs to be taken care of in backup or
>>> restore procedure.
>>> = Test Plan =
>>> Test scenarios that will be transformed to test cases for FreeIPA
>>> Continuous Integration during implementation or review phase.
>>> = RFE Author =
>>> [[User:Sbose|Sumit Bose]]
>>> _______________________________________________
>>> Freeipa-devel mailing list
>>> Freeipa-devel at redhat.com
>>> https://www.redhat.com/mailman/listinfo/freeipa-devel
>> -- 
>> Thank you,
>> Dmitri Pal
>> Sr. Engineering Manager IdM portfolio
>> Red Hat, Inc.
>> _______________________________________________
>> Freeipa-devel mailing list
>> Freeipa-devel at redhat.com
>> https://www.redhat.com/mailman/listinfo/freeipa-devel
> _______________________________________________
> Freeipa-devel mailing list
> Freeipa-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/freeipa-devel

Thank you,
Dmitri Pal

Sr. Engineering Manager IdM portfolio
Red Hat, Inc.

More information about the Freeipa-devel mailing list