<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Check the ns-slapd access and error logs of the DS instance hosting
the IPA instance. The strace output indicates that ns-slapd spent
most of its time waiting for network activity to happen (poll),
which is normal for ns-slapd.<br>
<br>
Jatin<br>
<div class="moz-cite-prefix">On 19/02/15 15:52, Jim Richard wrote:<br>
</div>
<blockquote
cite="mid:30BE1A3D-7170-485D-9A91-C868682450D9@placeiq.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
I’ve got 4 Redhat IDM masters in a multi-master
config. 3.0.0-42.el6.centos is the IPA version, 389-ds-base
version 1.2.11.15-48.el6_6, Centos 6.6
<div class=""><br class="">
</div>
<div class="">Monitoring established connections on port 389 and
dsInOps over time shows a consistent/even level of activity
however 2 of the 4 IPA servers show ever increasing CPI usage by
ns-slapd. One ns-slapd process will start to show increased CPU
for a time, then drop off as another then increases. This cycle
continues with each switch seeing more and more total SPU usage
by ns-slapd.</div>
<div class=""><br class="">
</div>
<div class="">strace timing for the offending ns-slapd looks like
the following:</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">% time seconds usecs/call calls errors
syscall</div>
<div class="">------ ----------- ----------- --------- ---------
----------------</div>
<div class=""> 96.12 9.342272 1133 8243
poll</div>
<div class=""> 3.86 0.375457 53 7066 41
futex</div>
<div class=""> 0.01 0.000668 0 8244 8244
getpeername</div>
<div class=""> 0.00 0.000374 0 929
close</div>
<div class=""> 0.00 0.000368 0 3201
read</div>
<div class=""> 0.00 0.000151 0 882
setsockopt</div>
<div class=""> 0.00 0.000095 2 42
access</div>
<div class=""> 0.00 0.000033 0 1365
fcntl</div>
<div class=""> 0.00 0.000000 0 42
open</div>
<div class=""> 0.00 0.000000 0 39
stat</div>
<div class=""> 0.00 0.000000 0 42
fstat</div>
<div class=""> 0.00 0.000000 0 1
madvise</div>
<div class=""> 0.00 0.000000 0 441
accept</div>
<div class=""> 0.00 0.000000 0 441
getsockname</div>
<div class=""> 0.00 0.000000 0 1
restart_syscall</div>
<div class="">------ ----------- ----------- --------- ---------
----------------</div>
<div class="">100.00 9.719418 30979 8285
total</div>
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">I have carefully reviewed cn=config settings on all
four master servers to confirm that they match.</div>
<div class=""><br class="">
</div>
<div class="">Based on this strace output can you perhaps point me
in the right direction, give me a clue on what I should be
looking at.</div>
<div class=""><br class="">
</div>
<div class="">Here’s a screen shot of my Zabbix reporting to help
describe the problem. Note the graph in the bottom right corner.</div>
<div class=""><br class="">
</div>
<div class="">The problem is most certainly related to replication
but I just don’t know what specifically to look at.</div>
<div class=""><br class="">
</div>
<div class=""><img apple-width="yes" apple-height="yes"
apple-inline="yes" id="1A427CEE-BFF2-4759-8335-AA62EF79AC68"
src="cid:part1.08030906.05000204@redhat.com" class=""
height="713" width="1212"></div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Thanks in advance for any clues you can provide.<br
class="">
<div class="">
<div style="color: rgb(0, 0, 0); font-family: Verdana;
font-size: 11px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br
class="Apple-interchange-newline">
<br class="">
</div>
<div style="color: rgb(0, 0, 0); font-family: Verdana;
font-size: 11px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
normal; orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<table style="color: rgb(34, 34, 34); margin: 0px;
font-size: 13.63636302948px; font-family: Times;
background-color: rgb(255, 255, 255);" class=""
cellpadding="0" cellspacing="0">
<tbody class="">
<tr class="" height="3">
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top;" class=""><br>
</td>
</tr>
</tbody><tbody class="">
<tr class="" height="3">
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(68,
54, 124);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(99,
80, 178);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(148,
145, 145);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(203,
203, 203);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(148,
145, 145);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(99,
80, 178);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(68,
54, 124);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(99,
80, 178);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(148,
145, 145);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(203,
203, 203);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(148,
145, 145);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(99,
80, 178);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class="" width="5"><br>
</td>
<td style="font-family: arial, sans-serif; margin:
0px; vertical-align: top; background-color: rgb(119,
117, 171);" class=""><br>
</td>
</tr>
</tbody><tbody class="">
<tr class="" height="10">
<td colspan="36" style="font-family: Georgia, serif;
margin: 0px; vertical-align: top; padding-top: 5px;
font-style: italic; font-size: 10px; letter-spacing:
0.1em; line-height: 1.5em; color: rgb(117, 107,
172);" class=""><span style="color: rgb(70, 70,
70);" class="">Jim Richard | </span><a
moz-do-not-send="true"
href="http://www.google.com/url?q=http%3A%2F%2Fwww.placeiq.com%2F&sa=D&sntz=1&usg=AFrqEzcYjZpDPyqW7feNK9EgLq-c9JlHiw"
target="_blank" style="color: rgb(117, 107, 172);
text-decoration: none;" class="">PlaceIQ</a> |
Systems Administrator | jrichard<a
moz-do-not-send="true"
href="mailto:name@placeiq.com" target="_blank"
style="color: rgb(117, 107, 172); text-decoration:
none;" class="">@placeiq.com</a> | <a
moz-do-not-send="true" style="color: rgb(17, 85,
204);" class="">+1 (646) 338-8905</a></td>
</tr>
</tbody>
</table>
<div class=""><br class="">
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
<br>
</body>
</html>