[Freeipa-users] freeipa master server disaster recovery

[root]

Rob Crittenden writes: 

> root wrote:
>>>> Greetings FreeIPA mailing list:
>>>> I have an FC11 environment setup for testing the FreeIPA implementation 
>>>> of kerberos+ldap w/admin utils.  Our primary purpose for kerberos right 
>>>> now is to provide auth services for coda.  However, once that gnat is 
>>>> squished, we'll of course be using kerberos for various other 
>>>> authentication services as well, and possibly using ldap for all manner 
>>>> of things (top of the list is basic server configuration information).
>>>> So far, FreeIPA is a wonderful product and has very much simplified our 
>>>> deployment.
>>>> My only real disappointment with FreeIPA, in fact, was seeing the 
>>>> notion of a "master server".  Moreover, I have not been able to 
>>>> determine what configuration or crucial data is stored on the master 
>>>> server -- of utmost importance, is _where_ said crucial 
>>>> configuration/data is stored so that we may suitably back it up.
>>>> This of course raises disaster recovery questions.  Such as, in the 
>>>> event of a disaster, is it possible (and advisable?) to somehow 
>>>> "promote" a FreeIPA slave/peer server to "master" status?  Or must we 
>>>> deploy a new server with the same name as the old and then somehow sync 
>>>> up the non-master data from the slave/peer(s)?  Obviously, the best 
>>>> scenario would be that we could do either, as the decision on whether 
>>>> to promote or re-deploy will depend heavily on circumstances 
>>>> surrounding the failure.
>>>> I am assuming the following scenario:
>>>> *) master server goes down
>>>> *) slave/peer(s) continue taking updates, the only exception being no 
>>>> F>> I think I had this all sorted, but then my laptop crashed.  Regardless, I 
>> can't seem to find it on my free-ipa server. 
>> 
> 
> rob
 

 

reeIPA servers may be deployed (correct??)
>>>> *) several days pass
>>>> *) master server is determined irreparable
>>>> At which point, what should we have done prior to this failure, to give 
>>>> us the most options for recovery?
>>>> Are there worse scenarios we can plan for?  Any other actions we can 
>>>> take that might save our bacon down the road?
>>> 
>>> Glad to hear the freeIPA is filling your needs.
>>> See this bug on promoting a new master: 
>>> https://bugzilla.redhat.com/show_bug.cgi?id=486950
>>> Each IPA server is in fact a complete standalone server. The only 
>>> distinctions about the "master" are:
>>> - It was the first one installed
>>> - It owns the CA private key so only it can generate replicas
>>> - It is the hub for replication
>>> The MMR we set up has each replica talking to this initial master. So if 
>>> it goes down all communication between other replicas is hosed.
>>> All is not lost though. You can use ipa-replica-manage to set up 
>>> additional communication links between your replicas. I don't think I'd 
>>> get too carried away with this though and make each replica talk to 
>>> every other replica.
>>> We just don't set up these additional links by default.
>> 
>> How can these links be established after the master is down for the 
>> count?
>> For that matter, how do we "convert" one of the non-master servers to a 
>> new master (assuming we've got the CA backed-up, of course)?
> 
> The bug outlines how to promote a replica to be the primary "master". You 
> basically just need to import the CA and setup the serial number file. 
> 
> So lets say you had a master and 2 replicas. In reality the only thing 
> that differentiates the first master is that it was installed first so has 
> the CA. As far as data replication goes there is no distinction, they are 
> all equal. 
> 
> So in this case your replication topology looks like: 
> 
>  A
> /\
> B C 
> 
> So all changes route through master A, the first installed server. If A 
> has died and won't be replaced then you need to tell B about C (or vice 
> versa) and tell it to forget about A. 
> 
> First you want to create an agreement from B to C:
> >> I think I had this all sorted, but then my laptop crashed.  Regardless, I 
>> can't seem to find it on my free-ipa server. 
>> 
> 
> rob
 

 


> # ipa-replica-manage add C 
> 
> Wait a bit to be sure all pending updates get flushed out, then delete the 
> agreement to A on both B and C: 
> 
> # ipa-replica-manage del C

Wow, thanks for the hand holding.  That's so easy it's embarrassing. 


>>> It all depends on your needs, paranoia, etc. Critically though you 
>>> should back up the CA cert and key and stick it in a vault somewhere, 
>>> the kerberos master key too if you're really paranoid (but it exists on 
>>> the replicas too, unlike the CA key).
>> 
>> Where is this crucial data stored?
> 
> IPA is basically in the job of herding cats so important information is 
> spread in a lot of places: 
> 
> - the CA private key is in /etc/dirsrv/slapd-INSTANCE
> - The file containing the last serial number issued by the CA is in 
> /var/lib/ipa/ca_serialno
> - The KDC still has important configuration in /var/kerberos
> - The DS itself is configured in /etc/dirsrv/slapd-INSTACE. This is where 
> all your data is stored
> - The configuration of Apache is important, that is in 
> /etc/httpd/conf.d/ipa.conf, /etc/httpd.conf/ipa-rewrite.conf
> - The Apache SSL database is probably reproducable in a pinch but it is in 
> /etc/httpd/alias
> - You might want the host keytabs in /etc/dirsrv/ds.keytab, 
> /etc/httpd/conf/ipa.keytab and perhaps the host keytab in 
> /etc/krb5.keytab.
> - If you've configured bind you'll probably want that configuration too. 
> 
> This may not be a an exhaustive list but it gives youa basic idea.

Certainly!  Question is, of the above data, assuming we're just rolling 
these servers out cookie cutter like and not futzing with each one 
individually, which files have irretrievable data (such as the CA private 
key and perhaps the CA serial).  We handle the keytab(s) already as we do 
extensive keytab based auth.  Most of the files listed seem like they are 
simply static config files, so I'm just double checking. 


Thanks!
 -Don
{void}