[Fedora-directory-users] db_verify

Tue Apr 17 01:08:27 UTC 2007

Ville Silventoinen wrote:

> Thanks Noriko, that explains a lot!
>
> I still have that other problem to solve, related to re-creating the 
> database (see the end of my first email). I will try to reproduce this 
> problem and create some test data this week.

Hi Ville,

> If I delete a database with the Console, it leaves behind couple of 
> index files:
>
> -rw-------  1 w3secure systems 16384 Mar 28 17:05 ancestorid.db4
> -rw-------  1 w3secure systems    18 Mar 28 17:03 DBVERSION
> -rw-------  1 w3secure systems 32768 Mar 28 17:05 id2entry.db4
>
> These index files don't seem to shrink when new entries are imported. 
> dbscan still shows the deleted entries in id2entry.
>
> I noticed a problem when I import a small set of entries, delete the 
> database, import large set of entries and if I query the entries, I 
> get the entries from the first set (they don't exist in the second 
> set). I can reproduce the problem. If I delete ancestorid.db4 and 
> id2entry.db4 manually when I delete the database, I don't have this 
> problem. Is there a reason why those two files are not deleted? Or can 
> this whole thing be caused by corrupted data?

I wonder this might be your case?

http://www.redhat.com/docs/manuals/dir-server/ag/7.1/entry_dist.html#22293
Deleting a Database
The following procedure describes deleting a directory database using 
the Directory Server Console. Deleting a database deletes the 
configuration information and entries for that database only, *not the 
physical database itself.*

http://www.redhat.com/docs/manuals/dir-server/ag/7.1/dbmanage.html#1117312
Importing a Database from the Console
When you perform an import operation from the Directory Server Console, 
an ldapmodify operation is executed to *append data*, as well as to 
modify and delete entries.

To overwrite the existing data, please take this step before importing 
the data.
http://www.redhat.com/docs/manuals/dir-server/ag/7.1/dbmanage.html#1117339
Initializing a Database from the Console

Or you could do the same thing from the command-line interface:
http://www.redhat.com/docs/manuals/dir-server/ag/7.1/dbmanage.html#1117378
Importing from the Command-Line
You can use three methods for importing data through the command-line:
. Using ldif2db - This import method overwrites the contents of your 
database and requires the server to be stopped.
. Using ldif2db.pl - This import method overwrites the contents of your 
database while the server is still running.

Hope it helps,
--noriko

>
> Thanks,
> Ville
>
>
> On Thu, 12 Apr 2007, Noriko Hosoi wrote:
>
>> Thank you, Ville, for the test data.  I could reproduce the db_verify 
>> problem.
>>
>> I have good news and bad news. :)  Good news, first...  Your db is 
>> not corrupted.  The error report from verify-db.pl is bogus.
>>
>> Bad news, next.  Please take a look at this bug.  We are going to 
>> provide a fixed utility some time soon.
>>
>> Summary: verify-db.pl (db_verify) does not work on a little endian 
>> machine
>>
>> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=236256
>>
>> Sorry about this inconvenience, and thank you for reporting the problem!
>> --noriko
>>
>> Ville Silventoinen wrote:
>>
>>> Hi Noriko,
>>>
>>> sorry it took so long to reply, I've been busy with other work.
>>>
>>> On Fri, 30 Mar 2007, Noriko Hosoi wrote:
>>>
>>>> Ville Silventoinen wrote:
>>>>
>>>>> I asked my manager but he doesn't think it's a good idea for 
>>>>> security reasons. The problem is that the data is our NIS 
>>>>> mail.aliases and passwd, and we don't want to distribute them to 
>>>>> the internet. He suggested I'll modify the data, so I can send a 
>>>>> sample to you. I'll do that next week.
>>>>
>>>> That would be great. Thanks! I'm interested in what type of 
>>>> characters your data contain. E.g., character set is UTF-8? Some of 
>>>> your DNs could contain any special characters such as '\'? etc...
>>>
>>>
>>> The character set should be plain ASCII. I created an imaginary 
>>> mail.aliases file. You can download it from here:
>>>
>>> http://www.ebi.ac.uk/systems-srv/mp/file-exchange/
>>>
>>> Type in "fedorads" to the Pass Phrase input box and click Go. You 
>>> should see three files: mail.aliases, mail.aliases.ldif and 
>>> 99user.ldif.
>>>
>>> I can reproduce my problem with the above files, for example, I've 
>>> tested like this:
>>>
>>> 1. Delete existing ebiRoot database (you could use userRoot).
>>> 2. Delete db/ebiRoot directory.
>>> 3. Create ebiRoot database.
>>> 4. Shutdown slapd.
>>> 5. Run db2index and verify-db.pl. No errors.
>>> 6. Start slapd.
>>> 7. Import mail aliases. I've tried with the Console and my own CLI, 
>>> which can import LDIF and add entries one-by-one. The method doesn't 
>>> seem to matter.
>>> 8. Shutdown slapd.
>>> 9. Run db2index and verify-db.pl, verify gives errors:
>>>
>>> Verify log files in db ... Good
>>> Verify db/ebiRoot/ancestorid.db4 ...
>>> DB ERROR: db_verify: Page 2: out-of-order key at entry 254
>>> DB ERROR: db_verify: DB->verify: db/ebiRoot/ancestorid.db4: 
>>> DB_VERIFY_BAD: Database verification failed
>>> Secondary index file ancestorid.db4 in db/ebiRoot is corrupted.
>>> Please run db2index(.pl) for reindexing.
>>> Verify db/ebiRoot/objectclass.db4 ...
>>> DB ERROR: db_verify: Page 2: out-of-order key at entry 255
>>> DB ERROR: db_verify: DB->verify: db/ebiRoot/objectclass.db4: 
>>> DB_VERIFY_BAD: Database verification failed
>>> Secondary index file objectclass.db4 in db/ebiRoot is corrupted.
>>> Please run db2index(.pl) for reindexing.
>>> Verify db/ebiRoot/nsuniqueid.db4 ... Good
>>> Verify db/ebiRoot/parentid.db4 ...
>>> DB ERROR: db_verify: Page 1: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: DB->verify: db/ebiRoot/parentid.db4: 
>>> DB_VERIFY_BAD: Database verification failed
>>> Secondary index file parentid.db4 in db/ebiRoot is corrupted.
>>> Please run db2index(.pl) for reindexing.
>>> Verify db/ebiRoot/cn.db4 ...
>>> DB ERROR: db_verify: Page 10: out-of-order key at entry 249
>>> DB ERROR: db_verify: DB->verify: db/ebiRoot/cn.db4: DB_VERIFY_BAD: 
>>> Database verification failed
>>> Secondary index file cn.db4 in db/ebiRoot is corrupted.
>>> Please run db2index(.pl) for reindexing.
>>> Verify db/ebiRoot/id2entry.db4 ... Good
>>> Verify db/ebiRoot/entrydn.db4 ... Good
>>> Verify db/ebiRoot/rfc822mailmember.db4 ...
>>> DB ERROR: db_verify: Page 2: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 3: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: DB->verify: db/ebiRoot/rfc822mailmember.db4: 
>>> DB_VERIFY_BAD: Database verification failed
>>> Secondary index file rfc822mailmember.db4 in db/ebiRoot is corrupted.
>>> Please run db2index(.pl) for reindexing.
>>>
>>>> So, in your ldif data, the mail attribute also has this type of 
>>>> value: "|/homes/majordom/wrapper 
>>>> stripmime.pl|/homes/majordom/wrapper resend -l foobar-dev 
>>>> foobar-dev-outgoing"?
>>>
>>>
>>> No, the People entries have a simpler mail value, like "foo at ebi.ac.uk".
>>>
>>>> And your mail index has the default indexing type: presence, 
>>>> equality, and substring?
>>>
>>>
>>> Yes.
>>>
>>>> What type of indexing does the rfc822MailMember attribute have?
>>>
>>>
>>> I've tried without any indexing, with presence and equality and with 
>>> presence, equality and substring. The above errors are from 
>>> verify-db.pl when I have presence and equality indeces. If I have 
>>> presence, equality and substring, I get these errors for 
>>> rfc822MailMember:
>>>
>>> Verify db/ebiRoot/rfc822mailmember.db4 ...
>>> DB ERROR: db_verify: Page 13: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 6: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 8: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 12: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 3: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 7: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 10: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 15: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 4: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 14: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 5: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 9: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: Page 11: unsorted duplicate set in sorted-dup 
>>> database
>>> DB ERROR: db_verify: DB->verify: db/ebiRoot/rfc822mailmember.db4: 
>>> DB_VERIFY_BAD: Database verification failed
>>> Secondary index file rfc822mailmember.db4 in db/ebiRoot is corrupted.
>>> Please run db2index(.pl) for reindexing.
>>>
>>>> Have we already heard what platform you are running the FDS on?
>>>
>>>
>>> CentOS release 4.4, Linux 2.6.9-42.ELsmp. Pentium III 2x1266MHz 
>>> CPUs, 2GB memory, SCSI disks. I'm using FDS 1.0.4.
>>>
>>> I'm away this week Wed-Fri, so I'll get back to you next week.
>>>
>>> Thanks for the help!
>>>
>>> Ville
>>>
>>> -- 
>>> Fedora-directory-users mailing list
>>> Fedora-directory-users at redhat.com
>>> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>>
>>
>>
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/fedora-directory-users/attachments/20070416/54ed6d37/attachment.bin>