<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 11/13/2014 03:02 AM, Walter van
Lille wrote:<br>
</div>
<blockquote
cite="mid:CAMqGCT_+K8ypO8970tVBoKAsGGO1qQ=bbV7BGe8YbkXcvG8BNQ@mail.gmail.com"
type="cite">
<div dir="ltr">Thanks Rich, I have installed the packages and run
gdb again. Hopefully the attached file is more useful.</div>
</blockquote>
<br>
The symbols are there. However, the server is almost completely
idle - no hangs, no deadlocks, no waiting on I/O.<br>
<br>
You must catch dirsrv while it is hung. You might want to run that
gdb script every few seconds during the time it appears to be hung.<br>
<br>
<blockquote
cite="mid:CAMqGCT_+K8ypO8970tVBoKAsGGO1qQ=bbV7BGe8YbkXcvG8BNQ@mail.gmail.com"
type="cite">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Nov 12, 2014 at 4:16 PM, Rich
Megginson <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:rmeggins@redhat.com" target="_blank">rmeggins@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span class="">
<div>On 11/12/2014 05:42 AM, Walter van Lille wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks again for the assistance guys. I
have saved two files and included it here. Hope it
tells you more than it does me.</div>
</blockquote>
<br>
</span> These stack traces contain no useful symbols.<br>
<br>
Please read <a moz-do-not-send="true"
href="http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes"
target="_blank">http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes</a>
to find out how to install the correct debuginfo packages
on your system. For IPA you will also need debuginfo
packages for ipa and slapi-nis<br>
<br>
If debuginfo-install is not working, see <a
moz-do-not-send="true"
href="https://www.centos.org/forums/viewtopic.php?t=1710"
target="_blank">https://www.centos.org/forums/viewtopic.php?t=1710</a><br>
<br>
If you simply cannot figure out how to install the
debuginfo packages, please email me with the following
information:<br>
<br>
$ rpm -q ipa-server slapi-nis 389-ds-base openldap db4 nss
nspr glibc<br>
<br>
and I will make them available to you<br>
<br>
NOTE: With debuginfo packages, the version of the
debuginfo package _must exactly match_ the version of the
associated package e.g. for <span>389-ds-base.x86_64
1.2.11.15-34.el6_5 you must have </span><br>
<span>389-ds-base-debuginfo.x86_64 1.2.11.15-34.el6_5<br>
<br>
If the versions do not match exactly, you will not get
the symbols.<br>
<br>
</span>
<div>
<div class="h5">
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Wallter</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Nov 11, 2014 at
8:03 PM, Rich Megginson <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:rmeggins@redhat.com"
target="_blank">rmeggins@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span>
<div>On 11/11/2014 10:37 AM, Martin Basti
wrote:<br>
</div>
<blockquote type="cite">
<div>On 11/11/14 15:58, Rich Megginson
wrote:<br>
</div>
<blockquote type="cite">
<div>On 11/11/2014 06:20 AM, Ludwig
Krispenz wrote:<br>
</div>
<blockquote type="cite"> <br>
<div>On 11/11/2014 02:14 PM, Martin
Basti wrote:<br>
</div>
<blockquote type="cite">
<div>Ludiwg (CCed) this seems like
old (fixed?) DS bug.<br>
</div>
</blockquote>
hmm, it says limit is 2097152, so it
already has the new setting, but the
error message says the packet is
800MB<b><br>
</b></blockquote>
<br>
<b>Right. That usually means the
server was expecting an encrypted
SASL buffer from the client, but
instead the client thinks SASL
encryption negotiation failed and
just sent a plain LDAP buffer. What
version of 389-ds-base are you
using? rpm -q 389-ds-base<br>
<br>
<a moz-do-not-send="true"
href="https://fedorahosted.org/389/ticket/47416"
target="_blank">https://fedorahosted.org/389/ticket/47416</a><br>
<br>
So, DO NOT increase your sasl io
buffer size - it will not fix the
problem, and it will leave you open
to DoS attacks.<br>
</b></blockquote>
<br>
He is using <br>
<div><br>
</div>
<div>CentOS release 6.5 (Final)<br>
</div>
<div>389-ds-base.x86_64
1.2.11.15-34.el6_5</div>
</blockquote>
<br>
<br>
</span> Ignore the "SASL encrypted packet
length exceeds maximum allowed limit"
error. All it means is the client has some
error and is canceling the connection.<br>
<br>
The bugzilla associated with <b><a
moz-do-not-send="true"
href="https://fedorahosted.org/389/ticket/47416"
target="_blank">47416</a> is targeted
for RHEL 7.1. The problem is that we were
never able to figure out what error the
client was getting that caused the client
to stop establishing the </b>SASL
encryption, and the original customer who
reported the problem dropped the case -
everything just started working, no further
errors. Note that 47416 doesn't really fix
the problem or address the root cause - the
root cause is some error on the client side
that causes it to stop doing encrypted SASL
I/O and send an LDAP unbind request.<br>
<br>
Let's get back to the actual problem - what
is the actual problem? The ldap server is
hanging or unresponsive? If so, then see <a
moz-do-not-send="true"
href="http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs"
target="_blank">http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs</a>.
Let's get some dirsrv stack traces during
the period of time when it appears to be
unresponsive.
<div>
<div><br>
<br>
<blockquote type="cite"> <br>
<blockquote type="cite"><b> <br>
</b>
<blockquote type="cite"><b> </b>
<blockquote type="cite">
<div> <br>
On 11/11/14 13:13, Walter van
Lille wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>I've just cleaned out
a ton of slapd_poll
timed out messages from
the output and changed
the names to protect the
innocent, :-)</div>
<div>Here is the output as
requested:</div>
<div><br>
</div>
<div><br>
</div>
<div><b>[05/Nov/2014:11:44:05
+0200] - SASL
encrypted packet
length exceeds maximum
allowed limit
(length=805565,
limit=2097152).
Change the
nsslapd-maxsasliosize
attribute in cn=config
to increase limit.</b></div>
</div>
<div><b><br>
</b></div>
<div><b>[10/Nov/2014:14:45:19
+0200] - slapd_poll(115)
timed out</b></div>
<div><b>[10/Nov/2014:14:45:19
+0200] sasl_io_enable -
Cannot enable SASL
security on connection
in CLOSING state</b></div>
<div><b>[10/Nov/2014:14:45:19
+0200] - Error: could
not add/remove IO layers
from connection</b></div>
<div>
<div><b>[11/Nov/2014:11:48:09
+0200] - slapd
shutting down -
signaling operation
threads</b></div>
<div><b>[11/Nov/2014:11:48:09
+0200] - slapd
shutting down -
waiting for 30 threads
to terminate</b></div>
</div>
<div>
<div><b>[11/Nov/2014:13:14:12
+0200] - slapd
shutting down -
closing down internal
subsystems and plugins</b></div>
<div><b>[11/Nov/2014:13:14:12
+0200] - Waiting for 4
database threads to
stop</b></div>
<div><b>[11/Nov/2014:13:14:13
+0200] - All database
threads now stopped</b></div>
<div><b>[11/Nov/2014:13:14:13
+0200] - slapd
stopped.</b></div>
<div><b>[11/Nov/2014:13:26:35
+0200] -
389-Directory/<a
moz-do-not-send="true"
href="http://1.2.11.15" target="_blank">1.2.11.15</a> B2014.219.179
starting up</b></div>
<div><b>[11/Nov/2014:13:26:35
+0200]
schema-compat-plugin -
warning: no entries
set up under
cn=computers,
cn=compat,dc=sample,dc=example</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Skipping CoS
Definition cn=Password
Policy,cn=accounts,dc=sample,dc=example--no
CoS Templates found,
which should be added
before the CoS
Definition.</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Skipping CoS
Definition cn=Password
Policy,cn=accounts,dc=sample,dc=example--no
CoS Templates found,
which should be added
before the CoS
Definition.</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - slapd
started. Listening on
All Interfaces port
389 for LDAP requests</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Listening on
All Interfaces port
636 for LDAPS requests</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Listening on
/var/run/slapd-SAMPLE-EXAMPLE.socket
for LDAPI requests</b></div>
<div><b>[11/Nov/2014:13:57:08
+0200] -
slapd_poll(78) timed
out</b></div>
</div>
<div><b><br>
</b></div>
<div><b><br>
</b></div>
<div><b><br>
</b></div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Tue, Nov 11, 2014 at
1:19 PM, Martin Basti <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mbasti@redhat.com" target="_blank">mbasti@redhat.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000">
<div>IMHO It's DS
bug, can you share
DS error log?<br>
pspacek CCed to
examine named
logs.<br>
<br>
Martin^2
<div>
<div><br>
<br>
On 11/11/14
12:13, Walter
van Lille
wrote:<br>
</div>
</div>
</div>
<div>
<div>
<blockquote
type="cite">
<div dir="ltr">Hi
Martin, thanks
for the reply.
<div>My
version:
bind-dyndb-ldap-2.3-5.el6.x86_64</div>
<div>The
server doesn't
have
journalctl
installed but
I have the
outputs from
the messages
and named.run
files that I
included here:</div>
<div><br>
</div>
<div>Messages:</div>
<div><br>
</div>
<div>
<div><b>Nov 11
12:30:13
freeipa
named[1481]:
error (network
unreachable)
resolving
'example.example.com.10.123.123.123/A/IN':
2001:500:2f::f#53</b></div>
<div><b>Nov 11
12:30:23
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
<div><b>Nov 11
12:30:23
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
<div><b>Nov 11
12:30:33
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
<div><b>Nov 11
12:30:33
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
</div>
<div><br>
</div>
<div>Named.run:</div>
<div><br>
</div>
<div>
<div><b>client
10.123.123.123#42639:
transfer of
'example.example/IN':
AXFR-style
IXFR started</b></div>
<div><b>client
10.123.123.123#42639:
transfer of
''example.example/IN':
AXFR-style
IXFR ended</b></div>
<div><b>client
10.123.123.123#46912:
transfer of
'10.123.123.123.in-addr.arpa/IN':
AXFR-style
IXFR started</b></div>
<div><b>client
10.123.123.123#46912:
transfer of
'10.123.123.123.in-addr.arpa/IN':
AXFR-style
IXFR ended</b></div>
<div><b>LDAP
query timed
out. Try to
adjust
"timeout"
parameter</b></div>
<div><b>LDAP
query timed
out. Try to
adjust
"timeout"
parameter</b></div>
<div><b>LDAP
query timed
out. Try to
adjust
"timeout"
parameter</b></div>
</div>
<div><br>
</div>
<div>I just
replaced the
IPs and the
actual names
with something
more generic.</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Walter</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Thu, Nov 6,
2014 at 5:00
PM, Martin
Basti <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:mbasti@redhat.com" target="_blank">mbasti@redhat.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
bgcolor="#FFFFFF"
text="#000000">
<div>
<div>
<div>On
06/11/14
14:58, Walter
van Lille
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>I need
some
assistance
please.</div>
<div>I've
taken over an
IPA server to
manage a few
months ago,
and it was
working fine
until recently
when it
started acting
up seemingly
off its own
accord.</div>
<div>When I do
an ipactl
status it
basically
gives an
output as
shown below:</div>
<div><br>
</div>
<div><br>
</div>
<div><b>Directory
Service:
RUNNING<br>
</b></div>
<div><b><br>
</b></div>
<div>
<div><b>Loooooooooooooooooooooooooooooooooooooooooooooooooong
pause... (To
the tune of 7
minutes
sometimes)</b></div>
</div>
<div><b><br>
</b></div>
<div>
<div><b>KDC
Service:
RUNNING</b></div>
<div><b>KPASSWD
Service:
RUNNING</b></div>
<div><b>DNS
Service:
RUNNING</b></div>
<div><b>MEMCACHE
Service:
RUNNING</b></div>
<div><b>HTTP
Service:
RUNNING</b></div>
<div><b>CA
Service:
RUNNING</b></div>
<div><b>ADTRUST
Service:
RUNNING</b></div>
<div><b>EXTID
Service:
RUNNING</b></div>
</div>
<div><br>
</div>
<div>Running
top showed
that ns-slapd
was munching
almost all my
resources, but
I got that
fixed by
upping the
cache.
Unfortunately
this did not
correct the
issue and it
still reacts
in the same
fashion,
although the
resources have
been freed up
now.</div>
<div>I've
noticed that
when I run dig
on either the
local server
or a remote
machine that
the query
basically just
times out as
shown here:</div>
<div><br>
</div>
<div>
<div> <b>dig
freeipa.myexample.sample</b></div>
<div><b><br>
</b></div>
<div><b>;
<<>>
DiG
9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1
<<>>
freeipa.myexample.sample</b></div>
<div><b>;;
global
options: +cmd</b></div>
<div><b>;;
connection
timed out; no
servers could
be reached</b></div>
</div>
<div><br>
</div>
<div>When the
KDC service
fails to
start, then
name lookups
seem OK, but
authentication
fails.
otherwise it's
dead in the
water.</div>
<div><br>
</div>
<div>This also
happens:</div>
<div>
<div><br>
</div>
<div><b>sudo
ipactl status</b></div>
<div><b>Directory
Service:
RUNNING</b></div>
<div><b>Unknown
error when
retrieving
list of
services from
LDAP:</b></div>
</div>
<div><b><br>
</b></div>
<div>My
software setup
is as follows:</div>
<div><br>
</div>
<div><b>CentOS
release 6.5
(Final)<br>
</b></div>
<div><b>389-ds-base.x86_64
1.2.11.15-34.el6_5<br>
</b></div>
<div><b>bind.x86_64
32:9.8.2-0.23.rc1.el6_5.1<br>
</b></div>
<div>
<div><b>bind-dyndb-ldap.x86_64</b></div>
<div><b>bind-libs.x86_64
32:9.8.2-0.23.rc1.el6_5.1</b></div>
<div><b>bind-utils.x86_64
32:9.8.2-0.23.rc1.el6_5.1</b></div>
<div><b>rpcbind.x86_64
0.2.0-11.el6
@anaconda-CentOS-201311291202.x86_64/6.5</b></div>
<div><b>samba4-winbind.x86_64</b></div>
</div>
<div><b>krb5-server.x86_64
1.10.3-15.el6_5.1<br>
</b></div>
<div><b><br>
</b></div>
<div><b>Linux
2.6.32-431.29.2.el6.x86_64
#1 SMP Tue Sep
9 21:36:05 UTC
2014 x86_64
x86_64 x86_64
GNU/Linux<br>
</b></div>
<div><br>
</div>
<div>It's not
a permanent
situation as
it sometimes
runs 100% for
a while, but
80% of the
time it is
unusable. If
anybody can
assist me,
please be so
kind.</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Walter</div>
<div><br>
</div>
</div>
</blockquote>
</div>
</div>
Hello please
which version
of
bind-dyndb-ldap
do you use?<br>
I had similar
issue with
bind-dyndb-ldap,
but it was
development
version, I'm
not sure if
this is your
case.<br>
When named was
failing,
dirserv was
really slow.<br>
<br>
Can you send
journalctl -b
-u named log
when dig
doesn't work??<span><font
color="#888888"><br>
<br>
<pre cols="72">--
Martin Basti</pre>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<br>
</div>
</div>
<span><font
color="#888888">
<pre cols="72">--
Martin Basti</pre>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<br>
<pre cols="72">--
Martin Basti</pre>
</blockquote>
<br>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
<br>
<pre cols="72">--
Martin Basti</pre>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
</div>
</div>
</div>
<br>
--<br>
Manage your subscription for the Freeipa-users
mailing list:<br>
<a moz-do-not-send="true"
href="https://www.redhat.com/mailman/listinfo/freeipa-users"
target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a><br>
Go To <a moz-do-not-send="true"
href="http://freeipa.org" target="_blank">http://freeipa.org</a>
for more info on the project<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>