[Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???

Fri May 11 21:53:35 UTC 2012

On 05/10/2012 10:54 PM, Rich Megginson wrote:
> On 05/10/2012 07:54 PM, David Copperfield wrote:
>> OK,
>>
>>  that means the steps below:
>>
>> 1) on IPA replica, lets create 4 IPA users: A,B,C and D.  Now make a
>> backup with 'db2ldif.pl -r ...'
>>
>> 2) on IPA replica, delete the user D. 'ipa user-del D'.
>>
>> 3, on IPA master, delete the user C. 'ipa user-del C'.
>>
>> 4, now check on other IPA master and IPA replica, both shows only two
>> users 'A' and 'B'. this is expected.
>>
>> 5, now on IPA replica, restore the backup with 'ldif2db.pl'
>>
>> 6, check on IPA replica immediately, 'ipa user-find' shows 4 users
>> 'A, B, C, D' at the beginning.
>>
>> 7, check IPA Master, 'ipa user-find' shows still only two users 'A, B'.
>>
>> 8, wait 3 minutes or so, check on IPA replica, and found that there
>> are only THREE users 'A, B, D'. The users 'C' is deleted now --
>> change propagated from IPA Master.
>>
>> 9, check on IPA Master again and again, there are still only two
>> users 'A, B'.
>>
>> 10, check on IPA Replica again and again, there are still three users
>> 'A, B,D'. --- this status is different from IPA Master's 'A,B', or
>> backup's  'A, B, C, D'.
>>
>>
>> If backup was created without '-r' option, then the step 8 above will
>> always show 'A,B,C,D', the same as backup.  with '-r' option make the
>> final result between.
>>
>>
>> Hope I have explained it clearly. Please advice something like
>> ipa2ldif.pl and ldif2ipa.pl tools. There are really the key useful
>> feature for serious production IPA deployment, which is definitely of
>> much higher priority than dogtag.
>
> Sounds like a bug.  What should happen is that the deletion of C and D
> should be propagated to replica.

Was a bug or a ticket filed?

>
>>
>> Thanks a lot.
>>
>> --David
>>
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Rich Megginson <rmeggins at redhat.com>
>> *To:* David Copperfield <cao2dan at yahoo.com>
>> *Cc:* E Deon Lackey <dlackey at redhat.com>; Petr Spacek
>> <pspacek at redhat.com>; Rob Crittenden <rcritten at redhat.com>;
>> "freeipa-users at redhat.com" <freeipa-users at redhat.com>
>> *Sent:* Thursday, May 10, 2012 6:37 PM
>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with
>> db2ldap.pl, ldap2db.pl ???
>>
>> On 05/10/2012 07:32 PM, David Copperfield wrote:
>>> Hi Rich and all,
>>>
>>> the '-r' option to db2ldif.pl <http://db2ldif.pl> doesn't work
>>> neither, it make few difference. 
>>>
>>> My command, backup and restore commands on the IPA replica are:
>>>
>>> db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com'
>>>
>>> ldif2db.pl <http://ldif2db.pl> -D 'cn=Directory Manager' -w - -i
>>> <the_backup_file_in_LDIF_format>
>>>
>>> The only difference is: after IPA master restart (restart happens
>>> after IPA replica's restore operation), the changes -- which applied
>>> on IPA master before backup -- are propagated to IPA replica. Which
>>> is in fact, make the restoration test end up with a result
>>> completely unusable on IPA replica, an result that is different from
>>> backup, and different from IPA master.
>>
>> I don't quite understand what you mean.
>>
>>>
>>> Please let me know if there are any other options/steps to follow.
>>> Thanks.
>>
>> Not sure what else to try.
>>
>>>
>>> --David
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>> *From:* Rich Megginson <rmeggins at redhat.com>
>>> <mailto:rmeggins at redhat.com>
>>> *To:* David Copperfield <cao2dan at yahoo.com> <mailto:cao2dan at yahoo.com>
>>> *Cc:* "freeipa-users at redhat.com" <mailto:freeipa-users at redhat.com>
>>> <freeipa-users at redhat.com> <mailto:freeipa-users at redhat.com>; Rob
>>> Crittenden <rcritten at redhat.com> <mailto:rcritten at redhat.com>; Petr
>>> Spacek <pspacek at redhat.com> <mailto:pspacek at redhat.com>
>>> *Sent:* Thursday, May 10, 2012 5:28 PM
>>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with
>>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ???
>>>
>>> On 05/10/2012 04:37 PM, David Copperfield wrote:
>>>> Hi Rich and all,
>>>>
>>>> Thanks for correction. They are db2ldif.pl <http://db2ldif.pl> and
>>>> ldif2db.pl <http://ldif2db.pl> scripts, which are originally for
>>>> 389 Directory Servers' backup and restore purposes. 
>>>>
>>>> There are no IPA tools for IPA system backup and restore. Is there
>>>> a plan to develop tools like ipa2ldif.pl <http://ipa2ldif.pl> and
>>>> ldif2ipa.pl <http://ldif2ipa.pl> soon? or, at least, whether it is
>>>> in IPA roadmap?
>>>>
>>>> For the second question: I use the simple way: ipa
>>>> user-add/user-delete/user-find to see whether data is propagated.
>>>> My testing steps are like this:
>>>>
>>>>  1, run 'ipa user-add testuser' on IPA replica, check it on IPA
>>>> master with 'ipa user-find testuser' and it is found in a few
>>>> seconds -- not 5 minutes.
>>>>
>>>>  2, run 'db2ldif.pl on IPA replica to save a backup.
>>>>
>>>>  3, run 'ipa user-del testuser' on IPA replica, then 'ipa
>>>> user-find' on IPA replica, and  it shows that the user is deleted.
>>>>
>>>>  4, double check 'ipa user-find test user' on IPA master, and it is
>>>> found deleted, which is as expected and it is propagated in just a
>>>> few seconds.
>>>>
>>>>  5, run 'ldif2db.pl' on the same IPA replica where the backup was
>>>> created.
>>>>
>>>>  6, run 'ipa user-find testuser' on IPA replica and it is found
>>>> that the user testuser is alive again.
>>>>
>>>>  7, run 'ipa user-find testuser' on IPA master. 1/3 times we can
>>>> find it -- and in just a few seconds. other 2/3 times it could not
>>>> be found even after HALF HOUR.
>>>>
>>>> Please have a quick duplicate tests at your side and advice what
>>>> normal users should do, because a reliable backup/restore solution
>>>> is definitely one of the key criteria. Thanks a lot.
>>>>
>>>
>>> Ok, I see.  The problem is that a regular db2ldif[.pl] does not save
>>> the replication meta-data.  You must use the -r option to generate
>>> an ldif file with the replication meta-data.  ldif2db[.pl] is
>>> destructive - it wipes out your database completely and replaces it,
>>> wiping out any replication meta-data in the process.  If you
>>> ldif2db[.pl] a file exported with db2ldif[.pl] -r, it will replace
>>> the replication meta-data too.
>>>
>>> See
>>> http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line
>>>
>>>> --David
>>>>  
>>>>
>>>>
>>>>  
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>> *From:* Rich Megginson <rmeggins at redhat.com>
>>>> <mailto:rmeggins at redhat.com>
>>>> *To:* David Copperfield <cao2dan at yahoo.com> <mailto:cao2dan at yahoo.com>
>>>> *Cc:* "freeipa-users at redhat.com" <mailto:freeipa-users at redhat.com>
>>>> <freeipa-users at redhat.com> <mailto:freeipa-users at redhat.com>; Rob
>>>> Crittenden <rcritten at redhat.com> <mailto:rcritten at redhat.com>; Petr
>>>> Spacek <pspacek at redhat.com> <mailto:pspacek at redhat.com>
>>>> *Sent:* Thursday, May 10, 2012 3:19 PM
>>>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with
>>>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ???
>>>>
>>>> On 05/10/2012 03:57 PM, David Copperfield wrote:
>>>>> Hi Rob, Petr and all,
>>>>>
>>>>> Because recently crashes of my IPA master and IPA replicas
>>>>> servers, I'm thinking of methods of backup/restore IPA user data:
>>>>> users, groups, host and server certificates etc.  
>>>>>
>>>>> It's said that the only official way is to create an extra IPA
>>>>> replica and backup/snapshot that replica all the way. But there
>>>>> still has a big chance that some mistakes propagate for a to whole
>>>>> IPA domain/realm before the IAP administrator find it and data got
>>>>> lost forever and some may not even be recovered.
>>>>>
>>>>> What I think is because both Dogtag and IPA store data in backend
>>>>> 389 directory servers separately, then if I freeze the change on
>>>>> one IPA replica for a few minutes first, then run db2ldap.pl
>>>>> <http://db2ldap.pl> for both 389 ldap backends, then un-freeze the
>>>>> IPA replica to get sync from master.
>>>>>
>>>>>  When data needs to be restored because of disasters, the backup
>>>>> files(in LDIF format -- for easy to read) can be restored to the
>>>>> two 389 LDAP backends on IPA replica with command ldap2db.pl
>>>>> <http://ldap2db.pl> during the freezing period.
>>>>
>>>> It's ldif2db.pl <http://ldif2db.pl> db2ldif.pl <http://db2ldif.pl>
>>>> not ldap
>>>>
>>>>>
>>>>>  Have anyone tried this solution yet? Is there any limitations?
>>>>>
>>>>> My experiences showed that the IPA replica did get data restored
>>>>> successfully (no dogtag is involved so only one LDAP backend is
>>>>> saved/restored). But the IPA master some times didn't get the data
>>>>> synced from IPA replica ( 1/3 times it is synced, 2/3 times needs
>>>>> manual command 'ipa-replica-manage force-sync  --from
>>>>> <ipaReplicaServer>' ).
>>>>
>>>> How did you verify that the data was synced?  Note that if a server
>>>> has been down for a while, it will take the supplier up to 5
>>>> minutes to recognize that the consumer is up again, without force sync.
>>>>
>>>>>
>>>>> Please shed a light in this area, as backup/restore of IPA
>>>>> master/replica is even not mentioned on the IPA document at all. 
>>>>>
>>>>> Thanks a lot.
>>>>>
>>>>> --David
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Freeipa-users mailing list
>>>>> Freeipa-users at redhat.com <mailto:Freeipa-users at redhat.com>
>>>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IPA project,
Red Hat Inc.

-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20120511/4bf974ae/attachment.htm>