[Pki-devel] replication of new/modified profiles

Wed Jul 2 17:17:23 UTC 2014

On Tue, 2014-07-01 at 12:42 +1000, Fraser Tweedale wrote:
> On Mon, Jun 30, 2014 at 02:28:09PM -0500, Endi Sukma Dewata wrote:
> > Hi, I have some comments about the design:
> > http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage
> > 
> > I think all system/default profiles should remain file-based and all custom
> > profiles should be LDAP-based. It will make a clean separation: system
> > profiles are owned by us (Dogtag developers), custom profiles are owned by
> > the admin.
> > 
> > The system profiles will be read-only. This way we will be able to update
> > the system profiles without writing any upgrade scripts because the files
> > will be updated automatically by RPM. Just one requirement, all server
> > instances must be upgraded to the same version.
> > 
> > If the admin wants to change a system profile, they can clone it into a
> > custom profile and make the changes there. The custom profiles cannot have
> > the same names as the system profiles, so there's won't be any
> > conflict/confusion, and no need to support a "restore" command. In general
> > we won't need to write upgrade scripts for custom profiles except if we
> > change the LDAP schema.
> > 
> 
> This was my original proposal, with the exception of making
> system/default profiles read-only and cloning them to make
> modifications.  I think this is a good modification to that
> proposal.
> 
> Ade was in favour of moving to LDAP only, but perhaps this variation
> makes using a mix of LDAP and file-based profiles more palatable?
> It would be good to hear from more people on this part of the design
> - to see if there's a clear winner.
> 

The problem with the above proposal is one of migration.  Customers are
used to both creating new profiles and modifying existing profiles to
suit their needs.  If we go along with this mechanism, we would have to
tell these customers that they would need to rename their existing
customized profiles (and update all the relevant clients etc.)

There is another problem, and that is that it is not clear that we want
updates to the default profiles to be propagated to existing instances.
I have looked at the profiles and there have been only a handful of
changes over the last 7 years.  Those changes include things like
updating the default signing algorithms or the default validity.  More
likely than not, admins would prefer that we not change the behavior of
profiles in existing instances underneath them.

The changes that I have found are all behavioral - and therefore things
that admin can opt out of -- or would prefer to do on their own
schedule.  There have been no structural changes.

If there are structural changes, then we need to (and can) provide an
upgrade script which would run with the automatic upgrade.  An example
of this would be a schema upgrade as we sort out how to represent
profiles in LDAP.

So, I think we should do the following:
1. Continue to provide the system profiles in files.  These files will
be parsed and stored in LDAP when an instance is created.

2. All profiles for an instance should live in LDAP.  This makes it
simple - no need to check to see if a profile is in LDAP or files or
both, and which has priority etc.  Tools will be provided to manage/
create/delete profiles etc.

3. Updates to system profile files will not affect the existing LDAP
profiles.  We can provide update scripts or manual instructions for
admins to run when they opt to do so.  This will be for behavioral
changes.

4. Structural changes will be done using upgrade scripts using the
database upgrade mechanism.  This framework is something that we had
planned to do already in 10.3.  We already have a model on how to do
this in our current upgrade framework.

> > About the schema itself, I suppose we want to have something that resembles
> > the actual Profile data structure (see ProfileData Java class). There should
> > be an LDAP attribute for each single-valued Java attribute (e.g. name,
> > description, enabled, visible). This way the profile is more manageable and
> > can be queried based on these attributes. For collection attributes (e.g.
> > inputs, outputs, policySets) we can use child LDAP entries to represent
> > them.
> > 
> > About the names, right now we use "profile" to refer to both cert profile in
> > CA and token profile in TPS. We probably should use separate terms to
> > distinguish them (e.g. CertProfile and TokenProfile). This applies to
> > Java/Python class names, REST interface, CLI, LDAP schema, etc.
> > 
> Good point.  I've s/profile/certProfile/ in the schema.
> 
> > So the LDAP entries for a profile may look like this:
> > 
> > dn: cn=MyUserCert,ou=Profiles,ou=CA,{suffix}
> > objectClass: certProfile
> > cn: MyUserCert
> > displayName: Manual User Dual-Use Certificate Enrollment
> > description: This certificate profile is for enrolling user certificates.
> > visible: true
> > enabled: true
> > enabledBy: admin
> > 
> > dn: ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,{suffix}
> > objectClass: organizationalUnit
> > ou: Inputs
> > 
> > dn: cn=i1,ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,o=pki-tomcat
> > objectClass: pkiCertProfileInput
> > cn: i1
> > classId: keyGenInputImpl
> > 
> 
> Introducing this schema (and the more fine-grained functionality it
> would support) could happen later.  I see the benefits but I think
> it's better to just get the profiles in LDAP as soon as possible,
> then look at how we can break them down to enable richer features.
> 
I would prefer to see if we could design something more fine grained at
first, rather than assuming an upgrade.  If nothing else, you will need
to separate out whether the profile is visible and enabled and by whom.

> Also, if we retain file-based profiles as you suggest, whatever
> queries LDAP makes easier, we would still need to be able to do
> those same queries across the file-based profiles.  So while
> file-based profiles remain, a finer-grained LDAP profile schema
> doesn't buy us anything in terms of reducing complexity or the
> amount of code to write.
> 
> > About the REST interface & CLI, since this will be the primary way to edit
> > profiles, we might want to have more granular commands to modify parts of
> > the profile. Right now with ca-profile-mod command you need to send the
> > entire profile in a file. It would be nice to be able to specify some
> > parameters to change certain attributes only, or use separate commands to
> > manage the inputs/outputs.
> > 
> > We'll also need an interface to find existing cert records that use a
> > certain profile and bulk modify them to use a different profile. This will
> > be useful when you create a clone to change the system profile.
> > 
> 
> Again, I agree with the benefits here, but feel that this may be out
> of scope for the initial changeover to LDAP profiles, given how much
> we need that for FreeIPA.
> 

Agreed. I think this can be a later development.

> > I'm not sure if we should support 10.2 -> 10.3 cloning. When we release 10.3
> > the 10.2 will still be fairly new so it might be reasonable to require all
> > clones to be upgraded. It will reduce the amount of testing requirement too.
> > 
> I'll incorporate this comment (and all your others) in the design
> proposal; you should see the updated document soon.
> 
> Cheers,
> 
> Fraser
> 
> > -- 
> > Endi S. Dewata
> > 
> > On 6/18/2014 2:44 AM, Fraser Tweedale wrote:
> > >Hi all,
> > >
> > >A requirement from the FreeIPA side is the ability to add and
> > >customise CA profiles.  Dogtag's current profile creation behaviour
> > >writes the new profile to the filesystem beside the standard
> > >profiles (as well as making the appropriate update to the registry,
> > >etc.)
> > >
> > >There does not seem to be a mechanism to distribute new/modified
> > >profiles to replicas - though perhaps I have missed something.
> > >
> > >Because this behaviour is required, unless I have overlooked
> > >something or there is a better way (in which case please shout out),
> > >I think it makes sense to begin a design proposal for an LDAP-based
> > >profile store.
> > >
> > >Finally, a brief mention of some tickets related to profile storage
> > >that could be good to tackle simultaneously should the proposed
> > >change go ahead:
> > >
> > >- https://fedorahosted.org/pki/ticket/778
> > >- https://fedorahosted.org/freeipa/ticket/4002
> > >
> > >_______________________________________________
> > >Pki-devel mailing list
> > >Pki-devel at redhat.com
> > >https://www.redhat.com/mailman/listinfo/pki-devel
> > >
> > 