[Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???

Fri May 11 02:54:57 UTC 2012

On 05/10/2012 07:54 PM, David Copperfield wrote:
> OK,
>
>  that means the steps below:
>
> 1) on IPA replica, lets create 4 IPA users: A,B,C and D.  Now make a 
> backup with 'db2ldif.pl -r ...'
>
> 2) on IPA replica, delete the user D. 'ipa user-del D'.
>
> 3, on IPA master, delete the user C. 'ipa user-del C'.
>
> 4, now check on other IPA master and IPA replica, both shows only two 
> users 'A' and 'B'. this is expected.
>
> 5, now on IPA replica, restore the backup with 'ldif2db.pl'
>
> 6, check on IPA replica immediately, 'ipa user-find' shows 4 users 'A, 
> B, C, D' at the beginning.
>
> 7, check IPA Master, 'ipa user-find' shows still only two users 'A, B'.
>
> 8, wait 3 minutes or so, check on IPA replica, and found that there 
> are only THREE users 'A, B, D'. The users 'C' is deleted now -- change 
> propagated from IPA Master.
>
> 9, check on IPA Master again and again, there are still only two users 
> 'A, B'.
>
> 10, check on IPA Replica again and again, there are still three users 
> 'A, B,D'. --- this status is different from IPA Master's 'A,B', or 
> backup's  'A, B, C, D'.
>
>
> If backup was created without '-r' option, then the step 8 above will 
> always show 'A,B,C,D', the same as backup.  with '-r' option make the 
> final result between.
>
>
> Hope I have explained it clearly. Please advice something like 
> ipa2ldif.pl and ldif2ipa.pl tools. There are really the key useful 
> feature for serious production IPA deployment, which is definitely of 
> much higher priority than dogtag.

Sounds like a bug.  What should happen is that the deletion of C and D 
should be propagated to replica.

>
> Thanks a lot.
>
> --David
>
>
>
> ------------------------------------------------------------------------
> *From:* Rich Megginson <rmeggins at redhat.com>
> *To:* David Copperfield <cao2dan at yahoo.com>
> *Cc:* E Deon Lackey <dlackey at redhat.com>; Petr Spacek 
> <pspacek at redhat.com>; Rob Crittenden <rcritten at redhat.com>; 
> "freeipa-users at redhat.com" <freeipa-users at redhat.com>
> *Sent:* Thursday, May 10, 2012 6:37 PM
> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with 
> db2ldap.pl, ldap2db.pl ???
>
> On 05/10/2012 07:32 PM, David Copperfield wrote:
>> Hi Rich and all,
>>
>> the '-r' option to db2ldif.pl <http://db2ldif.pl> doesn't work 
>> neither, it make few difference.
>>
>> My command, backup and restore commands on the IPA replica are:
>>
>> db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com'
>>
>> ldif2db.pl <http://ldif2db.pl> -D 'cn=Directory Manager' -w - -i 
>> <the_backup_file_in_LDIF_format>
>>
>> The only difference is: after IPA master restart (restart happens 
>> after IPA replica's restore operation), the changes -- which applied 
>> on IPA master before backup -- are propagated to IPA replica. Which 
>> is in fact, make the restoration test end up with a result completely 
>> unusable on IPA replica, an result that is different from backup, and 
>> different from IPA master.
>
> I don't quite understand what you mean.
>
>>
>> Please let me know if there are any other options/steps to follow. 
>> Thanks.
>
> Not sure what else to try.
>
>>
>> --David
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Rich Megginson <rmeggins at redhat.com> <mailto:rmeggins at redhat.com>
>> *To:* David Copperfield <cao2dan at yahoo.com> <mailto:cao2dan at yahoo.com>
>> *Cc:* "freeipa-users at redhat.com" <mailto:freeipa-users at redhat.com> 
>> <freeipa-users at redhat.com> <mailto:freeipa-users at redhat.com>; Rob 
>> Crittenden <rcritten at redhat.com> <mailto:rcritten at redhat.com>; Petr 
>> Spacek <pspacek at redhat.com> <mailto:pspacek at redhat.com>
>> *Sent:* Thursday, May 10, 2012 5:28 PM
>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with 
>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ???
>>
>> On 05/10/2012 04:37 PM, David Copperfield wrote:
>>> Hi Rich and all,
>>>
>>> Thanks for correction. They are db2ldif.pl <http://db2ldif.pl> and 
>>> ldif2db.pl <http://ldif2db.pl> scripts, which are originally for 389 
>>> Directory Servers' backup and restore purposes.
>>>
>>> There are no IPA tools for IPA system backup and restore. Is there a 
>>> plan to develop tools like ipa2ldif.pl <http://ipa2ldif.pl> and 
>>> ldif2ipa.pl <http://ldif2ipa.pl> soon? or, at least, whether it is 
>>> in IPA roadmap?
>>>
>>> For the second question: I use the simple way: ipa 
>>> user-add/user-delete/user-find to see whether data is propagated. My 
>>> testing steps are like this:
>>>
>>>  1, run 'ipa user-add testuser' on IPA replica, check it on IPA 
>>> master with 'ipa user-find testuser' and it is found in a few 
>>> seconds -- not 5 minutes.
>>>
>>>  2, run 'db2ldif.pl on IPA replica to save a backup.
>>>
>>>  3, run 'ipa user-del testuser' on IPA replica, then 'ipa user-find' 
>>> on IPA replica, and  it shows that the user is deleted.
>>>
>>>  4, double check 'ipa user-find test user' on IPA master, and it is 
>>> found deleted, which is as expected and it is propagated in just a 
>>> few seconds.
>>>
>>>  5, run 'ldif2db.pl' on the same IPA replica where the backup was 
>>> created.
>>>
>>>  6, run 'ipa user-find testuser' on IPA replica and it is found that 
>>> the user testuser is alive again.
>>>
>>>  7, run 'ipa user-find testuser' on IPA master. 1/3 times we can 
>>> find it -- and in just a few seconds. other 2/3 times it could not 
>>> be found even after HALF HOUR.
>>>
>>> Please have a quick duplicate tests at your side and advice what 
>>> normal users should do, because a reliable backup/restore solution 
>>> is definitely one of the key criteria. Thanks a lot.
>>>
>>
>> Ok, I see.  The problem is that a regular db2ldif[.pl] does not save 
>> the replication meta-data.  You must use the -r option to generate an 
>> ldif file with the replication meta-data.  ldif2db[.pl] is 
>> destructive - it wipes out your database completely and replaces it, 
>> wiping out any replication meta-data in the process.  If you 
>> ldif2db[.pl] a file exported with db2ldif[.pl] -r, it will replace 
>> the replication meta-data too.
>>
>> See 
>> http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line
>>
>>> --David
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>> *From:* Rich Megginson <rmeggins at redhat.com> 
>>> <mailto:rmeggins at redhat.com>
>>> *To:* David Copperfield <cao2dan at yahoo.com> <mailto:cao2dan at yahoo.com>
>>> *Cc:* "freeipa-users at redhat.com" <mailto:freeipa-users at redhat.com> 
>>> <freeipa-users at redhat.com> <mailto:freeipa-users at redhat.com>; Rob 
>>> Crittenden <rcritten at redhat.com> <mailto:rcritten at redhat.com>; Petr 
>>> Spacek <pspacek at redhat.com> <mailto:pspacek at redhat.com>
>>> *Sent:* Thursday, May 10, 2012 3:19 PM
>>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with 
>>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ???
>>>
>>> On 05/10/2012 03:57 PM, David Copperfield wrote:
>>>> Hi Rob, Petr and all,
>>>>
>>>> Because recently crashes of my IPA master and IPA replicas servers, 
>>>> I'm thinking of methods of backup/restore IPA user data: users, 
>>>> groups, host and server certificates etc.
>>>>
>>>> It's said that the only official way is to create an extra IPA 
>>>> replica and backup/snapshot that replica all the way. But there 
>>>> still has a big chance that some mistakes propagate for a to whole 
>>>> IPA domain/realm before the IAP administrator find it and data got 
>>>> lost forever and some may not even be recovered.
>>>>
>>>> What I think is because both Dogtag and IPA store data in backend 
>>>> 389 directory servers separately, then if I freeze the change on 
>>>> one IPA replica for a few minutes first, then run db2ldap.pl 
>>>> <http://db2ldap.pl> for both 389 ldap backends, then un-freeze the 
>>>> IPA replica to get sync from master.
>>>>
>>>>  When data needs to be restored because of disasters, the backup 
>>>> files(in LDIF format -- for easy to read) can be restored to the 
>>>> two 389 LDAP backends on IPA replica with command ldap2db.pl 
>>>> <http://ldap2db.pl> during the freezing period.
>>>
>>> It's ldif2db.pl <http://ldif2db.pl> db2ldif.pl <http://db2ldif.pl> 
>>> not ldap
>>>
>>>>
>>>>  Have anyone tried this solution yet? Is there any limitations?
>>>>
>>>> My experiences showed that the IPA replica did get data restored 
>>>> successfully (no dogtag is involved so only one LDAP backend is 
>>>> saved/restored). But the IPA master some times didn't get the data 
>>>> synced from IPA replica ( 1/3 times it is synced, 2/3 times needs 
>>>> manual command 'ipa-replica-manage force-sync  --from 
>>>> <ipaReplicaServer>' ).
>>>
>>> How did you verify that the data was synced?  Note that if a server 
>>> has been down for a while, it will take the supplier up to 5 minutes 
>>> to recognize that the consumer is up again, without force sync.
>>>
>>>>
>>>> Please shed a light in this area, as backup/restore of IPA 
>>>> master/replica is even not mentioned on the IPA document at all.
>>>>
>>>> Thanks a lot.
>>>>
>>>> --David
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Freeipa-users mailing list
>>>> Freeipa-users at redhat.com  <mailto:Freeipa-users at redhat.com>
>>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>>
>>>
>>>
>>
>>
>>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20120510/419807c5/attachment.htm>