[Freeipa-users] Please help: How to restore IPA Master/Replicas from daily IPA Replica setup???

Tue May 22 17:26:59 UTC 2012

Dale Macartney wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> cc'ing group list back in for other opinions.
>
> On 05/22/2012 03:38 PM, Rich Megginson wrote:
>> On 05/22/2012 08:36 AM, Dmitri Pal wrote:
>>> On 05/22/2012 10:10 AM, Rich Megginson wrote:
>>>> On 05/22/2012 04:38 AM, Dmitri Pal wrote:
>>>>> On 05/22/2012 04:28 AM, Dale Macartney wrote:
>>>>>> Dmitri, Rob
>>>>>>
>>>>>> I thought I might reply to you both directly, just in case others on
>>>>>> the list vent frustrations on the ongoing discussion of this topic.
>>>>>>
>>>>>> I've been reading through the archives of the list for hot backup
>>>>>> solutions, and this email thread really stood out. I am seeing a
>>>>>> general consensus of backing up everything, and in some cases, even
>>>>>> backing up a virtualized guest disk image to maintain a backup. I
>>>>>> personally feel this is the wrong message people should be getting
>>>>>> into their heads about a DR solution for restoring IPA.
>>>>>>
>>>>>> I was wondering, and feel free to correct me here if you see fit, if
>>>>>> it would be beneficial to have a similar method of backing up IPA (and
>>>>>> replicas), in a similar fashion to how Microsoft recommend their
>>>>>> Domain Controllers to be backed up. A "system state backup" of sorts.
>>>>>> Where a backup is performed on all Domain Controllers (or in our case,
>>>>>> IPA servers). Basically, resulting in an individual restore point for
>>>>>> each replica. From here, you have an entire backup, which will only
>>>>>> ever bee used for that ONE server it was intended for. Essentially a
>>>>>> complete dump and load approach.
>>>>>>
>>>>>> It is best practice in a Windows environment to perform these backups
>>>>>> several times a week in small non-changing environments. So my
>>>>>> thinking is, if we have a "daily backup" solution which could be used
>>>>>> to run on each replica or master, then this should suffice in an
>>>>>> adequate procedure to give to customers.
>>>>>>
>>>>>> In short, I'm more than happy to put my hand up on this one to help
>>>>>> free up your time. I can easily take this on with a few of the lads
>>>>>> here in the UK and get some customer feed back from mates within my
>>>>>> former employment who are quite well versed in the realms of IPA.
>>>>>>
>>>>>> Would this be of any help to you? Do you see this as the right
>>>>>> direction to take on this matter? I'd love to hear your thoughts
>>>>>>
>>>>>> Rhys, Gav, cc'ing you in on this one. I'd like to throw this onto our
>>>>>> running list of IPA integration projects.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Dale
>>>>> First of all thank you for the offer!
>>>>>
>>>>> It seems that there are two main use cases:
>>>>> 1) Catastrophic failure
>>>>> 2) Data deletion
>>>>>
>>>>> In the case of the catastrophic failure you want to have all
>>>>> data+configuration+keys backed up to be able to effectively start over
>>>>> and re-install/recover from the backup.
> could we not have the ability of restoring only specific data? Like most
> backup solutions?
>
> for example
> having a utility where you can run "ipa backup all" could cover the
> data+config+keys, however depending on a catastrophic failure or data
> deletion, maybe have something along the lines of "ipa restore data" if
> we simply wanted to restore the data element of the backup.
>
> Thoughts? IMO, i think we should look for a KISS method which is
> specific to the application stack at hand.

Yes, this is the approach we're taking, as they are really separate 
problems.

We aren't too keen on trying to back up individual files becuase IPA 
touches so many things, in so many different configurations, that the 
possibility that some important file is missed is rather high. The 
impact could mean a restored server that doesn't work.

At this point we're recommending full backups and restores for 
system-level backups.

As for data we're still working on it. Being able to identify and 
restore certain users/groups, etc is something we want but haven't yet 
worked out the details.

>>>>> In this case IMO having a VM approach like the one JR uses is a viable
>>>>> solution. Rob, Simo, Rich do you agree?
>>>> We would need to test this to make sure VM snapshots don't cause
>>>> problems with replication and/or kerberos since those are sensitive to
>>>> time. All of the testing we have ever done for RHDS/389 for
>>>> backup/restore is based on simple database binary and ldif backups.
>>>> We've never had to take into account restoring to a filesystem time in
>>>> the past or a VM state that is in the past.
>>> Why in the past?
>>> If you take snapshots regularly say every other day when you restart a
>>> VM it should act as if the connection to it was lost for couple days.
>>
>> Ok. Maybe it will work just fine. All I'm saying is that we better test
> this well with a number of replication scenarios.
> Should we really be considering snapshots as an "IPA" method of
> recovery? That puts a prerequisite on the customer using either SAN
> snapshotting or virtualization technologies.
> If I use RHEV 2.2 as an example here, RHEV was not adopted by many
> customers because there was a prerequisite of Windows. If we have a
> reliance on other technologies for such key recovery principles this
> will undoubtedly (possibly more in the short than long term) have a
> knock on effect of adoption.
>
> Yes it might work, but should we be sending this message across? In the
> short term, if it works then great, however I think this should give us
> insight into a more robust method in the future.
>
>>
>>> I do not see how the Kerberos and time are related here.
>>>
>>>>> In the case of data deletion we one needs to keep LDAP data around and
>>>>> not necessarily all the configuration and keys. And not even all the
>>>>> data needs to be backed up.
>>>>>
>>>>> So there should probably be two different procedures.
>>>>>
>>>>> I also think that creating a VM snapshot and recovering from such
>>>>> snapshot can be automated. We should probably provide some kind of
>>>>> script or command to do so.
>>>>> We stayed a bit shy from either procedure because the set of the config
>>>>> files IPA touches changes all the time and until we get it stable it
>>>>> would be hard to commit to a flawless backup. We feel we will miss
>>>>> something and will get bugs that we will have to address. Yeah in the
>>>>> current state there is a bit of hand-waving... May be it is time to
>>>>> identify the set of of all things we touch on the system and try to have
>>>>> a more solid set of procedures and recommendations and bite the bullet
>>>>> if we miss something.
>>>>>
>>>>> Now back to your offer, first of all which of the two use cases you
>>>>> think you would be able to contribute to?
>>>>> Do you think it makes sense to script something for either of the cases?
>>>>> What? Would you be able to help?
>>>>>
>>>>> JR offered help too. He is waiting for the blessing to describe a VM
>>>>> based approach. So first we should decide if it is the right way to go.
>>>>> For the first case IMO it is but I want other opinions.
>>>>>
>>>>> And one more thing. May be we should continue this thread on the list so
>>>>> please reply there.
>>>>>
>>>
>>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQIcBAEBAgAGBQJPu7mGAAoJEAJsWS61tB+q5MkP+wYr0nK5u0BzWV0Ar7dE7EXj
> mC+oxwv3Q+H7fluDXgd7r+HeLhAQU0i+FmVLraQHWvoNN99l11s5RtChu6dPaICW
> ThgPP/k85uNUyiBJysePgB/xv+VTaRJ9SZVMPxecLBE72U7RZHAWDRAvYS9yU1Dv
> 4qVB9KOQdvTrxjOITH7XEDx9LbZmNQ2ViWOxQTbHY23v6t1VRXmgIRcWtKcfBgJd
> sq69fOA4pXV6YfHnk/gYoT5TIclZnv/qUzKPEGI/XvPuohzLjsdnfsoEQZpXzSxz
> L+U7sVYDGSq5EoOKEmFT4MdHEG7niGUbYLzIx8RD+uzXbPtI11BECe/esGF7pYkJ
> U4yxrnxPMxSmja9hccPephdNodmbBht4t4UKuFBVOfOC1N/yhOgys6MH4bbQmOMR
> dN9/fLJAVVdIjMiytsCNvFXH4rM44nJ9wzW5xziUx+472qjFjUn7Jwudp3n1+fB5
> w5JrY0R1315tLS4Bxhqhzx9wQFALytKgilxJhx2DW1RghQTk6YcFg7fF1ELXfJsH
> YcnOEbzcv20vwEcFTb8ilejvJorZACGTbKg9U3oMoz5WQEoMg448m6SL2Rp7J8+0
> dFUyA78HzklbqPBgAGgWdXQ9ZKR+oUVigYEynZ4pUxKH7KWDitOtZHKUlxx1BFKq
> n/BzLu9/FUEsMvOcsp23
> =ENst
> -----END PGP SIGNATURE-----
>