[Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
Rich Megginson
rmeggins at redhat.com
Mon May 14 18:36:51 UTC 2012
On 05/11/2012 04:05 PM, David Copperfield wrote:
> Please feel free to do it. Thanks.
Done. https://fedorahosted.org/389/ticket/369
feel free to add yourself to the CC list, and supply any more details
>
> --David
>
> ------------------------------------------------------------------------
> *From:* Dmitri Pal <dpal at redhat.com>
> *To:* Rich Megginson <rmeggins at redhat.com>
> *Cc:* David Copperfield <cao2dan at yahoo.com>; Rob Crittenden
> <rcritten at redhat.com>; E Deon Lackey <dlackey at redhat.com>;
> "freeipa-users at redhat.com" <freeipa-users at redhat.com>
> *Sent:* Friday, May 11, 2012 2:53 PM
> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with
> db2ldap.pl, ldap2db.pl ???
>
> On 05/10/2012 10:54 PM, Rich Megginson wrote:
>> On 05/10/2012 07:54 PM, David Copperfield wrote:
>>> OK,
>>>
>>> that means the steps below:
>>>
>>> 1) on IPA replica, lets create 4 IPA users: A,B,C and D. Now make a
>>> backup with 'db2ldif.pl <http://db2ldif.pl> -r ...'
>>>
>>> 2) on IPA replica, delete the user D. 'ipa user-del D'.
>>>
>>> 3, on IPA master, delete the user C. 'ipa user-del C'.
>>>
>>> 4, now check on other IPA master and IPA replica, both shows only
>>> two users 'A' and 'B'. this is expected.
>>>
>>> 5, now on IPA replica, restore the backup with 'ldif2db.pl
>>> <http://ldif2db.pl>'
>>>
>>> 6, check on IPA replica immediately, 'ipa user-find' shows 4 users
>>> 'A, B, C, D' at the beginning.
>>>
>>> 7, check IPA Master, 'ipa user-find' shows still only two users 'A, B'.
>>>
>>> 8, wait 3 minutes or so, check on IPA replica, and found that there
>>> are only THREE users 'A, B, D'. The users 'C' is deleted now --
>>> change propagated from IPA Master.
>>>
>>> 9, check on IPA Master again and again, there are still only two
>>> users 'A, B'.
>>>
>>> 10, check on IPA Replica again and again, there are still three
>>> users 'A, B,D'. --- this status is different from IPA Master's
>>> 'A,B', or backup's 'A, B, C, D'.
>>>
>>>
>>> If backup was created without '-r' option, then the step 8 above
>>> will always show 'A,B,C,D', the same as backup. with '-r' option
>>> make the final result between.
>>>
>>>
>>> Hope I have explained it clearly. Please advice something like
>>> ipa2ldif.pl <http://ipa2ldif.pl> and ldif2ipa.pl
>>> <http://ldif2ipa.pl> tools. There are really the key useful feature
>>> for serious production IPA deployment, which is definitely of much
>>> higher priority than dogtag.
>>
>> Sounds like a bug. What should happen is that the deletion of C and
>> D should be propagated to replica.
>
> Was a bug or a ticket filed?
>
>>
>>>
>>> Thanks a lot.
>>>
>>> --David
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>> *From:* Rich Megginson <rmeggins at redhat.com>
>>> <mailto:rmeggins at redhat.com>
>>> *To:* David Copperfield <cao2dan at yahoo.com> <mailto:cao2dan at yahoo.com>
>>> *Cc:* E Deon Lackey <dlackey at redhat.com>
>>> <mailto:dlackey at redhat.com>; Petr Spacek <pspacek at redhat.com>
>>> <mailto:pspacek at redhat.com>; Rob Crittenden <rcritten at redhat.com>
>>> <mailto:rcritten at redhat.com>; "freeipa-users at redhat.com"
>>> <mailto:freeipa-users at redhat.com> <freeipa-users at redhat.com>
>>> <mailto:freeipa-users at redhat.com>
>>> *Sent:* Thursday, May 10, 2012 6:37 PM
>>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with
>>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ???
>>>
>>> On 05/10/2012 07:32 PM, David Copperfield wrote:
>>>> Hi Rich and all,
>>>>
>>>> the '-r' option to db2ldif.pl <http://db2ldif.pl> doesn't work
>>>> neither, it make few difference.
>>>>
>>>> My command, backup and restore commands on the IPA replica are:
>>>>
>>>> db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com'
>>>>
>>>> ldif2db.pl <http://ldif2db.pl> -D 'cn=Directory Manager' -w - -i
>>>> <the_backup_file_in_LDIF_format>
>>>>
>>>> The only difference is: after IPA master restart (restart happens
>>>> after IPA replica's restore operation), the changes -- which
>>>> applied on IPA master before backup -- are propagated to IPA
>>>> replica. Which is in fact, make the restoration test end up with a
>>>> result completely unusable on IPA replica, an result that is
>>>> different from backup, and different from IPA master.
>>>
>>> I don't quite understand what you mean.
>>>
>>>>
>>>> Please let me know if there are any other options/steps to follow.
>>>> Thanks.
>>>
>>> Not sure what else to try.
>>>
>>>>
>>>> --David
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>> *From:* Rich Megginson <rmeggins at redhat.com>
>>>> <mailto:rmeggins at redhat.com>
>>>> *To:* David Copperfield <cao2dan at yahoo.com> <mailto:cao2dan at yahoo.com>
>>>> *Cc:* "freeipa-users at redhat.com" <mailto:freeipa-users at redhat.com>
>>>> <freeipa-users at redhat.com> <mailto:freeipa-users at redhat.com>; Rob
>>>> Crittenden <rcritten at redhat.com> <mailto:rcritten at redhat.com>; Petr
>>>> Spacek <pspacek at redhat.com> <mailto:pspacek at redhat.com>
>>>> *Sent:* Thursday, May 10, 2012 5:28 PM
>>>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with
>>>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ???
>>>>
>>>> On 05/10/2012 04:37 PM, David Copperfield wrote:
>>>>> Hi Rich and all,
>>>>>
>>>>> Thanks for correction. They are db2ldif.pl <http://db2ldif.pl> and
>>>>> ldif2db.pl <http://ldif2db.pl> scripts, which are originally for
>>>>> 389 Directory Servers' backup and restore purposes.
>>>>>
>>>>> There are no IPA tools for IPA system backup and restore. Is there
>>>>> a plan to develop tools like ipa2ldif.pl <http://ipa2ldif.pl> and
>>>>> ldif2ipa.pl <http://ldif2ipa.pl> soon? or, at least, whether it is
>>>>> in IPA roadmap?
>>>>>
>>>>> For the second question: I use the simple way: ipa
>>>>> user-add/user-delete/user-find to see whether data is propagated.
>>>>> My testing steps are like this:
>>>>>
>>>>> 1, run 'ipa user-add testuser' on IPA replica, check it on IPA
>>>>> master with 'ipa user-find testuser' and it is found in a few
>>>>> seconds -- not 5 minutes.
>>>>>
>>>>> 2, run 'db2ldif.pl on IPA replica to save a backup.
>>>>>
>>>>> 3, run 'ipa user-del testuser' on IPA replica, then 'ipa
>>>>> user-find' on IPA replica, and it shows that the user is deleted.
>>>>>
>>>>> 4, double check 'ipa user-find test user' on IPA master, and it
>>>>> is found deleted, which is as expected and it is propagated in
>>>>> just a few seconds.
>>>>>
>>>>> 5, run 'ldif2db.pl' on the same IPA replica where the backup was
>>>>> created.
>>>>>
>>>>> 6, run 'ipa user-find testuser' on IPA replica and it is found
>>>>> that the user testuser is alive again.
>>>>>
>>>>> 7, run 'ipa user-find testuser' on IPA master. 1/3 times we can
>>>>> find it -- and in just a few seconds. other 2/3 times it could not
>>>>> be found even after HALF HOUR.
>>>>>
>>>>> Please have a quick duplicate tests at your side and advice what
>>>>> normal users should do, because a reliable backup/restore solution
>>>>> is definitely one of the key criteria. Thanks a lot.
>>>>>
>>>>
>>>> Ok, I see. The problem is that a regular db2ldif[.pl] does not
>>>> save the replication meta-data. You must use the -r option to
>>>> generate an ldif file with the replication meta-data. ldif2db[.pl]
>>>> is destructive - it wipes out your database completely and replaces
>>>> it, wiping out any replication meta-data in the process. If you
>>>> ldif2db[.pl] a file exported with db2ldif[.pl] -r, it will replace
>>>> the replication meta-data too.
>>>>
>>>> See
>>>> http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line
>>>>
>>>>> --David
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>> *From:* Rich Megginson <rmeggins at redhat.com>
>>>>> <mailto:rmeggins at redhat.com>
>>>>> *To:* David Copperfield <cao2dan at yahoo.com>
>>>>> <mailto:cao2dan at yahoo.com>
>>>>> *Cc:* "freeipa-users at redhat.com" <mailto:freeipa-users at redhat.com>
>>>>> <freeipa-users at redhat.com> <mailto:freeipa-users at redhat.com>; Rob
>>>>> Crittenden <rcritten at redhat.com> <mailto:rcritten at redhat.com>;
>>>>> Petr Spacek <pspacek at redhat.com> <mailto:pspacek at redhat.com>
>>>>> *Sent:* Thursday, May 10, 2012 3:19 PM
>>>>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with
>>>>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ???
>>>>>
>>>>> On 05/10/2012 03:57 PM, David Copperfield wrote:
>>>>>> Hi Rob, Petr and all,
>>>>>>
>>>>>> Because recently crashes of my IPA master and IPA replicas
>>>>>> servers, I'm thinking of methods of backup/restore IPA user data:
>>>>>> users, groups, host and server certificates etc.
>>>>>>
>>>>>> It's said that the only official way is to create an extra IPA
>>>>>> replica and backup/snapshot that replica all the way. But there
>>>>>> still has a big chance that some mistakes propagate for a to
>>>>>> whole IPA domain/realm before the IAP administrator find it and
>>>>>> data got lost forever and some may not even be recovered.
>>>>>>
>>>>>> What I think is because both Dogtag and IPA store data in backend
>>>>>> 389 directory servers separately, then if I freeze the change on
>>>>>> one IPA replica for a few minutes first, then run db2ldap.pl
>>>>>> <http://db2ldap.pl> for both 389 ldap backends, then un-freeze
>>>>>> the IPA replica to get sync from master.
>>>>>>
>>>>>> When data needs to be restored because of disasters, the backup
>>>>>> files(in LDIF format -- for easy to read) can be restored to the
>>>>>> two 389 LDAP backends on IPA replica with command ldap2db.pl
>>>>>> <http://ldap2db.pl> during the freezing period.
>>>>>
>>>>> It's ldif2db.pl <http://ldif2db.pl> db2ldif.pl <http://db2ldif.pl>
>>>>> not ldap
>>>>>
>>>>>>
>>>>>> Have anyone tried this solution yet? Is there any limitations?
>>>>>>
>>>>>> My experiences showed that the IPA replica did get data restored
>>>>>> successfully (no dogtag is involved so only one LDAP backend is
>>>>>> saved/restored). But the IPA master some times didn't get the
>>>>>> data synced from IPA replica ( 1/3 times it is synced, 2/3 times
>>>>>> needs manual command 'ipa-replica-manage force-sync --from
>>>>>> <ipaReplicaServer>' ).
>>>>>
>>>>> How did you verify that the data was synced? Note that if a
>>>>> server has been down for a while, it will take the supplier up to
>>>>> 5 minutes to recognize that the consumer is up again, without
>>>>> force sync.
>>>>>
>>>>>>
>>>>>> Please shed a light in this area, as backup/restore of IPA
>>>>>> master/replica is even not mentioned on the IPA document at all.
>>>>>>
>>>>>> Thanks a lot.
>>>>>>
>>>>>> --David
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Freeipa-users mailing list
>>>>>> Freeipa-users at redhat.com <mailto:Freeipa-users at redhat.com>
>>>>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>
>
> --
> Thank you,
> Dmitri Pal
>
> Sr. Engineering Manager IPA project,
> Red Hat Inc.
>
>
> -------------------------------
> Looking to carve out IT costs?
> www.redhat.com/carveoutcosts/ <http://www.redhat.com/carveoutcosts/>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20120514/2691c6fd/attachment.htm>
More information about the Freeipa-users
mailing list