[Freeipa-users] lock table errors

[Andy Thompson]

> >> On 02/23/2016 03:02 PM, Andy Thompson wrote:
> >>> Came across one of my replicas this morning with the following in
> >>> the error log
> >>>
> >>> [20/Feb/2016:17:23:38 -0500] - libdb: BDB2055 Lock table is out of
> >>> available lock entries
> >>> [20/Feb/2016:17:23:38 -0500] entryrdn-index - _entryrdn_delete_key:
> >>> Deleting C1 failed; Cannot allocate memory(12)
> >>> [20/Feb/2016:17:23:38 -0500] - database index operation failed BAD
> >>> 1031, err=12 Cannot allocate memory
> >>> [20/Feb/2016:17:23:38 -0500] -
> >>> index_del_entry(changenumber=1328662,cn=changelog, 0x26) failed
> (12)
> >>> [20/Feb/2016:17:23:38 -0500] DSRetroclPlugin - delete_changerecord:
> >>> could not delete change record 1328662 (rc: 1)
> >>> [20/Feb/2016:17:23:38 -0500] - libdb: BDB2055 Lock table is out of
> >>> available lock entries
> >>> [20/Feb/2016:17:23:38 -0500] entryrdn-index - _entryrdn_get_elem:
> >>> Failed to position cursor at the key: 1328666: Cannot allocate
> >>> memory(12)
> >>> [20/Feb/2016:17:23:38 -0500] entryrdn-index - _entryrdn_delete_key:
> >>> Failed to position cursor at the key: 1328666: Cannot allocate
> >>> memory(12)
> >>> [20/Feb/2016:17:23:38 -0500] - libdb: BDB2055 Lock table is out of
> >>> available lock entries
> >>> [20/Feb/2016:17:23:38 -0500] - database index operation failed BAD
> >>> 1031, err=12 Cannot allocate memory
> >>> [20/Feb/2016:17:23:38 -0500] -
> >>> index_del_entry(changenumber=1328663,cn=changelog, 0x26) failed
> (12)
> >>> [20/Feb/2016:17:23:38 -0500] NSMMReplicationPlugin - changelog
> >>> program
> >>> - _cl5CompactDBs: failed to compact
> >>> 5f1d2b12-cf1411e4-b055ba8a-f4b484f7; db error - 12 Cannot allocate
> >>> memory
> >>> [20/Feb/2016:17:23:38 -0500] DSRetroclPlugin - delete_changerecord:
> >>> could not delete change record 1328663 (rc: 1)
> >>> [20/Feb/2016:17:23:41 -0500] ldbm_back_delete - conn=0 op=0 [retry:
> >>> 1]
> >> No original_tombstone for changenumber=1330335,cn=changelog!!
> >>> And then nothing.  Was troubleshooting some clients that were having
> >> issues resolving some trusted domain users.
> >>> I restarted IPA and it rolled through a few thousand missing change
> >>> records
> >>>
> >>> 23/Feb/2016:08:39:34 -0500] DSRetroclPlugin - delete_changerecord:
> >>> could not delete change record 1328696 (rc: 32)
> >>>
> >>> Any thoughts as to what might have caused the lock table errors?
> >> in BerkeleyDB this means that the number of pages which would have to
> >> be locked in one transaction exceeds the configured number of locks.
> >> This could happen if eg a large group is deleted and for each member
> >> of the group inside the same transaction the memberof attribute has
> >> to be modified
> >
> > Are there any configuration options to increase that setting?  And would it
> have caused the replica to become unresponsive?
> you can change
> 
> nsslapd-db-locks
> 
> in the entry:
> 
> dn: cn=config,cn=ldbm database,cn=plugins,cn=config
> 
> yes. in that state it would not process updates, the txn should be finally
> aborted and the system should recover,but ..

Is there any rule of thumb or anything I can look at to get an idea of what I should increase that to or should it even be necessary?

The current setting has a default of 10000

cn=database,cn=monitor,cn=ldbm database,cn=plugins,cn=config

currently shows 

nsslapd-db-current-locks: 82

What might cause that to spike up that significantly to deplete the locks?  That's a pretty huge jump.

Running 389-ds-base-1.3.4.0-26.el7_2.x86_64

-andy