[Freeipa-users] Purge old entries in /var/lib/dirsrv/slapd-xxx/cldb/xxx.db4 file

Wed Jan 27 11:54:08 UTC 2016

Hi,

> Hi,

On 12/22/2015 11:43 AM, David Goudet wrote:

>>    Hi,

>>    I have multimaster replication environment. On each replica, folder /var/lib/dirsrv/slapd-xxxx/cldb/ has big size (3~GB) and old entries in /var/lib/dirsrv/slapd-xxx/cldb/xxx.db4 have three month year old:

>>    sudo dbscan -f /var/lib/dirsrv/slapd-xxxx/cldb/ef155b03-dda611e2-a156db20-90xxx06_51c9aed900xxxxxx000.db4 | less
    dbid: 56239e5e000000040000
             replgen: 1445174777 Sun Oct 18 15:26:17 2015
             csn: 56239e5e000000040000
             uniqueid: e55d5e01-26f211e4-9b60db20-90c3b706
             dn: xxxx
             operation: modify
                     krbLastSuccessfulAuth: 20151018132617Z
                     modifiersname: cn=Directory Manager
                     modifytimestamp: 20151018132617Z
                     entryusn: 68030946

>>    My questions are:

>>    a) How to purge old entries in file /var/lib/dirsrv/slapd-xxx/cldb/xxx.db4? (what is the procedure)
>>    b) What is the right configuration to limit increase of this file?

> setting changelog maxage should be sufficient to trim changes, but the age is not the only condition deciding if a recored in the changelog can be deleted. - for each replicaID the last record will never be deleted, independent of its age, so if you have replicas in your topology which are not (or not frequently) updated directly there will be old changes in the changelog - if the replica where the trimming is run and if it has replication agreements to other replicas, changes which were not yet replicated to the other replica will not be purged. So, if you have some stale agreements to other replicas this could prevent trimming as well.

> Also trimming removes changelog records and frees space internally ro th edb4 file to be reused, but it will not shrink the file size

Thank you for your response. I agree with you, to identify where the problem is i enabled the errors logs: nsslapd-errorlog-level: 8192

And i found these errors:

[23/Dec/2015:09:46:40 +0100] agmt="cn=meTo<remote IPA>" (ds01:389) - load=1 rec=69 csn=567a5a43000100040000
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): replay_update: Sending modify operation (
dn="fqdn=xxx.xxx.xxx,cn=computers,cn=accounts,dc=xxx,dc=xxx" csn=567a5a43000100040000)
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): replay_update: modifys operation (dn="fqd
n=pad01.xxx.xxx.xxx,cn=computers,cn=accounts,dc=xxx,dc=xxx" csn=567a5a43000100040000) not sent - empty
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): replay_update: Consumer successfully sent operation with csn 567a5a43000100040000
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): Skipping update operation with no message_id (uniqueid 25791707-b72211e2-a156db20-90c3b706, CSN 567a5a43000100040000):
...
23/Dec/2015:09:46:40 +0100] agmt="cn=meTo<remote IPA server>" (ds01:389) - load=1 rec=72 csn=567a5a44000000040000
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): replay_update: Sending modify operation (dn="fqdn=xxx
x.xxx.xxx,cn=computers,cn=accounts,dc=xxx,dc=xxx" csn=567a5a44000000040000)
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): replay_update: modifys operation (dn="fqdn=xxxx
xxx,cn=computers,cn=accounts,dc=xxx,dc=xxx" csn=567a5a44000000040000) not sent - empty
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): replay_update: Consumer successfully sent operation with csn 567a5a44000000040000
[23/Dec/2015:09:46:40 +0100] NSMMReplicationPlugin - agmt="cn=meTo<remote IPA server>" (ds01:389): Skipping update operation with no message_id (uniqueid 7cfafb01-7fc711e4-974fdb20-90c3b706, CSN 567a5a44000000040000):

Replication between the two master/master IPA server seems to work well, but we can see many skipped requests:

repl-monitor -r -c xxx -w                                                                                      
<hr width=90% size=3><br>
Enter password for (:): 
<center><p><font class=page-subtitle color=#0099cc>Time Lag Legend:</font><p>
<table cellpadding=6 cols=3 width=40%>
<tr>

<td bgcolor=#ccffcc><center>within 5 min</center></td>

<td bgcolor=#ffffcc><center>within 60 min</center></td>

<td bgcolor=#ffcccc><center>over 60 min</center></td>

<td bgcolor=red><center>server n/a</center></td>
</table></center>
<p><p><hr><p>

<p><center class=page-subtitle><font color=#0099cc>
Master:&nbsp <a href="ldap://xxxx:389/">xxxx:389</a></center>

<p><table border=0 cellspacing=1 cellpadding=6 cols=10 width=100% class=bgColor9>

<tr><td colspan=10><center>
<font class=areatitle>Replica ID: </font><font class=text28>3</font>
<font class=areatitle>Replica Root: </font><font class=text28>dc=xxxx,dc=xxx</font>
<font class=areatitle>Max CSN: </font><font class=text28>56a8ad14000200030000 (01/27/2016 12:42:12 2 0)</font>

<tr class=bgColor16>
<th nowrap>Receiver</th>
<th nowrap>Time Lag</th>
<th nowrap>Max CSN</th>
<th nowrap>Last Modify Time</th>
<th nowrap>Supplier</th>
<th nowrap>Sent/Skipped</th>
<th nowrap>Update Status</th>
<th nowrap>Update Started</th>
<th nowrap>Update Ended</th>
<th nowrap colspan=2>Schedule</th>
<th nowrap>SSL?</th>
</tr>

tr class=bgColor13>                                                                                                                                 
<td rowspan=1 width=5% class=bgColor5><a href="ldap://xxxx:389/">xxx:389</a><BR>Type: master</td>
<td rowspan=1 width=5% nowrap bgcolor=#ccffcc><center>- 0:44:30</center></td>
<td rowspan=1 width=15% nowrap>56a8a2a6000100030000<br>(01/27/2016 11:57:42 1 0)</td>
<td rowspan=1 width=15% nowrap>1/27/2016 11:56:01</td>
<td width=5% nowrap><center>xxxx:389</center></td>
<td width=3% nowrap>3429 / 4188985195</td>
<td width=20% nowrap>0 Replica acquired successfully: Incremental update succeeded</td>
<td nowrap>01/27/2016 12:40:31</td>
<td nowrap>01/27/2016 12:40:32</td>
<td colspan=2 width=10% nowrap>always in sync</td>
<td width=3% nowrap class=bgColor5>SASL/GSSAPI</td>
</table>
<p><p><hr><p>

<p><center class=page-subtitle><font color=#0099cc>
Master:&nbsp <a href="ldap://xx:389/">xxx:389</a></center>

<p><table border=0 cellspacing=1 cellpadding=6 cols=10 width=100% class=bgColor9>

<tr><td colspan=10><center>
<font class=areatitle>Replica ID: </font><font class=text28>4</font>
<font class=areatitle>Replica Root: </font><font class=text28>dc=xxxx,dc=xxxx</font>
<font class=areatitle>Max CSN: </font><font class=text28>56a8ad1b000100040000 (01/27/2016 12:42:19 1 0)</font>

<tr class=bgColor16>
<th nowrap>Receiver</th>
<th nowrap>Time Lag</th>
<th nowrap>Max CSN</th>
<th nowrap>Last Modify Time</th>
<th nowrap>Supplier</th>
<th nowrap>Sent/Skipped</th>
<th nowrap>Update Status</th>
<th nowrap>Update Started</th>
<th nowrap>Update Ended</th>
<th nowrap colspan=2>Schedule</th>
<th nowrap>SSL?</th>
</tr>

tr class=bgColor13>
<td rowspan=1 width=5% class=bgColor5><a href="ldap://xxxx:389/">xxx:389</a><BR>Type: master</td>
<td rowspan=1 width=5% nowrap bgcolor=#ccffcc><center>- 0:15:07</center></td>
<td rowspan=1 width=15% nowrap>56a8a990000500040000<br>(01/27/2016 12:27:12 5 0)</td>
<td rowspan=1 width=15% nowrap>1/27/2016 12:25:32</td>
<td width=5% nowrap><center>xxxx:389</center></td>
<td width=3% nowrap>2434 / 3284152884</td>
<td width=20% nowrap>0 Replica acquired successfully: Incremental update started</td>
<td nowrap>01/27/2016 12:40:38</td>
<td nowrap>n/a</td>
<td colspan=2 width=10% nowrap>always in sync</td>
<td width=3% nowrap class=bgColor5>SASL/GSSAPI</td>
</table>

Questions
----

Is these observertions (request not sent and skipped request) can explain the problem? If yes how to fix it?
If no how to get information to identify the problem? 

Thank you for your help

David