[Freeipa-devel] Multi-master replication with puppet

[Simo Sorce]

On Fri, 2014-06-06 at 14:06 -0400, James wrote:
> On Fri, 2014-06-06 at 08:51 -0400, Simo Sorce wrote:

> > Yes, the dm_password was chosen because it is needed to actually
> > initialize and install the replica, so instead of asking it twice we
> > just ask for it once and use it *also* to encrypt the prepared file. But
> > the password is still currently absolutely necessary.
> Okay... But it's only needed for encryption on replica-prepare side,
> right?

We use it to encrypt the package and also to check that it is indeed the
correct password by performing a successful bind to LDAP.
I think we may be using it also to generate the certificates for the
replica, but I haven't looked at that code for a while and may be
remembering incorrectly.
 
> > But let me ask a more important question, how do you distribute the
> > public keys securely ? Is it puppet fetching them from each
> > replica-to-be server and sending them to the first master server ?
> Yes. That is one approach.

What did you have in mind ?

> > It is also needed to actually store the DM password on the replica
> > server itself in it's own cn=Directory Manager super user entry.
> > Although I think we could send the hash for that.
> This is what I was thinking... I'll need to know how to extract this,
> send it (encrypted), and use it instead of --password

It might be greppable out of the dse.ldif file I think.

> > >  I would need some help to find an
> > > alternative for this process. As was suggested, perhaps a one-time
> > > password could be generated and sent across the wire (with puppet) to
> > > the replica. Any mechanism that doesn't need the dm_password would be
> > > great.
> > 
> > Forget custom schemes.
> > 
> > The simplest way to handle this (which I wanted to do for a while) is to
> > pre-create the replica's LDAP principal and keys at replica-prepare time
> > so we can start right away with GSSAPI authentication for the first
> > replication and create agreements directly using GSSAPI auth. This would
> > in turn simplify replication agreements creation as we won't need
> > anymore to do the dance where we create the agreement with a plain text
> > bind and then we need to convert the agreement to use GSSAPI.
> 
> This of course sounds fine to me. As long as it meets my criteria of not
> needing the password, it should work. Is this an easy patch? IOW
> something that I could test a build of within the next month at the
> latest?

Well, you may not need a password (you do because you need to be DM on
the replica to create replication agreements, but we can do some magic
there and use ldapi as root I guess) but you'll need a keytab, which is
a secret, just like a password and must not be exposed to anyone and
anything just like the DM password. But that wouldn't be a problem, the
keytab would be in the encrypted prepared file.

> > > I don't think there is any other need for the dm_password in
> > > ipa-replica-install. Did I miss something?
> > 
> > See above, it needs the password to actually create the DM account at DS
> > setup time. *also* the DM password is used to setup some parts of the
> > dogtag CA so if any of your replica is also a CA clone you go straight
> > back to needing the DM password.
> Which I understand we could get around by passing around the (salted?)
> hashed password, correct?

Not for the CA, unfortunately the CA requires the DM password in the
clear.

> > > Of course if I store the dm_password in puppet, then it's easy to do
> > > this peering because it will be easy to use the password on each host.
> > > I could be done my puppet module already :) But I'm going out of my
> > > way to do it right, so that you'll all be proud of the work too!
> > 
> > Ok, here is a quickest method to do what you want to do IMO:
> > 
> > First of all, you need a DM password for the initial server and you need
> > to store it somewhere because the admin must know what it is, there is
> > no way around this.
> > So I guess you are planning to store it in a file under /root in the
> > master server or something like that. This means puppet has access to
> > the DM password w/o storing it in the manifest.
> Wrong. The whole idea is I want to keep puppet's grubby hands off the
> password. I generate the password on the host, encrypt it with the
> admin's public key, and never store it unencrypted. More details here:
> https://ttboj.wordpress.com/2014/06/06/securely-managing-secrets-for-freeipa-with-puppet/

Sure, I do not mean that puppet needs to transport it in the clear, but
it gets to know it in the abstract, either because it generates it (on
the ipa server) or because it transports it (encrypted) and unpacks it
in the replicas.
But the fact is a certain points it must have access to it to do the
initial install.

> > Use the DM password to do everything as usual AND create the asymmetric
> > keys in the replicas and *securely* transmit them to the master.
> Keep in mind I'm only transmitting public keys, so it doesn't have to be
> secure as long as I know verify the identity.

Bzzzzzzzzt. How do I know that "these are the public key I am looking
for" if they are not transmitted securely ?
If someone can sneak in an additional key or replace a public key in
transit now it gets access to everything as soon you transmit the
encrypted package!

> > Now wrap the current replica file (yes it is already encrypted, who
> > cares), AND the DM password into a new envelope encrypted with your
> > public keys.
> > 
> > Presto! Now you transfer the blob back to the replicas, and each replica
> > can be scripted, and puppet has no passwords in its manifest.
> 
> This approach won't work because the first axiom is incorrect. To do
> this, I would have to keep the dm_password around on each host with a
> key that could be read and used by anyone with root. So effectively I
> might as well store the password in puppet.

Look, at the moment there is no way around that, you have to have the DM
password *or* an equivalent secret to install replicas, there is simply
no way around that. You need to transport some form of credentials from
the master ipa server to the replicas because the replicas need to
authenticate to the master in order to be allowed to become replicas.
It is as simple as that, whether it is the DM password, or a keytab, or
something else.

> > > *symmetry*
> > > 
> > > It would be especially elegant and beneficial to FreeIPA and the
> > > puppet module if the process of "peering" was symmetrical, that is to
> > > say, similar for any host. If I could use ipa-server-install to setup
> > > N hosts, and then run a secondary command to cause certain ones to
> > > "join" this would make the process much more natural for puppet.
> > 
> > This is intrinsically not possible.
> That is unfortunate. I could hack a version of this where by each host
> runs the installer, but the ones that need to peer start by running:
> 
> ipa-server-install --uninstall && ipa-replica-install
> 
> but that is less elegant that the solution existing natively.

It is not about being inelegant, it is about being useless to me.

> > > In addition, this would ensure that the configuration management itself
> > > is HA. Without this type of functionality, then if the first ipa
> > > server isn't available, then config management will be blocked.
> > 
> > Yeah, chicken-egg issues of bootstrapping infrastructures. I currently
> > do not see any way around this.
> 
> See my above idea for a dirty hack. (--uninstall)

install; uninstall; replica-install ... the first 2 steps are a no-op.

> The point is that nothing in your config management process (think 100%
> automatic without human intervention) can stall. Even if something
> fails, there's a path by which it can continue, even if it means wiping
> out a server. Highly available config management is necessary for
> certain futures.

Sure, if the first server install fails you wipe and restart, where is
the problem ?

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York