[Fedora-directory-users] MMR broken, reinitialization erases db

[Chris St. Pierre]

With the help of a couple folks on IRC (thanks richm, uffe!), here's
what I figured out I can do:

In order to get two machines doing MMR again, I first got rid of any
MMR agreements between them, and then shut them both down.  Then I
chose one and exported the LDAP database with:

/opt/fedora-ds/slapd-instance/db2ldif -n userRoot

I copied the LDIF file to the other node.  Then I imported it on both:

/opt/fedora-ds/slapd-instance/ldif2db -n userRoot -i /
     /opt/fedora-ds/slapd-instance/ldif/2007_03_22_141131.ldif

Then I went into the changelogdb/ folder and blew away all of the
__db.*, *.db4, and log.* files.  At this point, I started Fedora DS on
both nodes again.  I then used mmr.pl to re-initialize the MMR
agreement between the two of them, and all was well.

I've now got MMR working again between three nodes; the fourth will
get added back in late tonight.

This may be more cautious than is necessary, but it's working.

I still have no clue what caused this initially, but I don't really
care (unless it happens again).

Thanks for everyone's help!

Chris St. Pierre
Unix Systems Administrator
Nebraska Wesleyan University

On Thu, 22 Mar 2007, Chris St. Pierre wrote:

> Sometime earlier this week (still trying to determine when), the
> multi-master replication on one of our databases broke.  I tried to
> reinitialize it between a few of the hosts, and I got a bunch of
> errors:
>
> [22/Mar/2007:09:27:39 -0500] NSMMReplicationPlugin -
> multimaster_be_state_change: replica o=isp is going offline; disabling
> replication
> [22/Mar/2007:09:27:41 -0500] - WARNING: Import is running with
> nsslapd-db-private-import-mem on; No other process is allowed to
> access the database
> [22/Mar/2007:09:27:45 -0500] - ERROR bulk import abandoned
> [22/Mar/2007:09:27:45 -0500] - import userRoot: Aborting all import
> threads...
> [22/Mar/2007:09:27:53 -0500] - import userRoot: Import threads
> aborted.
> [22/Mar/2007:09:27:53 -0500] - import userRoot: Closing files...
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/owner.db4: unable to
> flush: No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/mail.db4: unable to
> flush: No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/modifytimestamp.db4:
> unable to flush: No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/telephoneNumber.db4:
> unable to flush: No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/nsUniqueId.db4: unable
> to flush: No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/objectclass.db4: unable
> to flush:No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/ou.db4: unable to
> flush: No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/icsCalendar.db4: unable
> to flush:No such file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/sambaSID.db4: unable to
> flush: Nosuch file or directory
> [22/Mar/2007:09:27:56 -0500] - libdb: userRoot/givenName.db4: unable
> to flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/gidnumber.db4: unable
> to flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/createtimestamp.db4:
> unable to flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/cn.db4: unable to
> flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/sn.db4: unable to
> flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/uid.db4: unable to
> flush: No suchfile or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/uidNumber.db4: unable
> to flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/aci.db4: unable to
> flush: No suchfile or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/uniquemember.db4:
> unable to flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/parentid.db4: unable to
> flush: Nosuch file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/entrydn.db4: unable to
> flush: No such file or directory
> [22/Mar/2007:09:27:57 -0500] - libdb: userRoot/id2entry.db4: unable to
> flush: Nosuch file or directory
> [22/Mar/2007:09:27:57 -0500] - import userRoot: Import failed.
> [22/Mar/2007:09:27:57 -0500] - process_bulk_import_op: NULL backend
>
> This erased the database, and I was left with no data.  Subsequently,
> I've restarted FDS, restored from backup using bak2db.pl, and it still
> doesn't work.
>
> Any ideas?
>
> Chris St. Pierre
> Unix Systems Administrator
> Nebraska Wesleyan University
> ----------------------------
> Never send mail to thobrux at nebrwesleyan.edu
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>
>