<div dir="ltr">Roughly 128 clients.. But when the services start going gaa-gaa it also causes time-outs with the naming service running on the same server. I may be wrong, but wouldn't that basically kill any chance of a client connecting to the server anyway? I'm just lucky that the clients aren't really there for general usage, so the only times that users actually try to log in is when it is support staff that needs to check or restart a pretty static, stable service.<div><br></div><div>I just want to add that the server is not hung when doing an ipactl status and the KDC service is not started as below, so are we barking up the right tree looking at the DS service? (PS, the KDC service was not manually stopped by me.<br><div><br></div><div><b>sudo ipactl status</b></div><div><b><br></b></div><div><div><b>Directory Service: RUNNING</b></div><div><b>KDC Service: STOPPED</b></div><div><b>KPASSWD Service: RUNNING</b></div><div><b>DNS Service: RUNNING</b></div><div><b>MEMCACHE Service: RUNNING</b></div><div><b>HTTP Service: RUNNING</b></div><div><b>CA Service: RUNNING</b></div><div><b>ADTRUST Service: RUNNING</b></div><div><b>EXTID Service: RUNNING</b></div></div><div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 13, 2014 at 12:27 PM, Ludwig Krispenz <span dir="ltr"><<a href="mailto:lkrispen@redhat.com" target="_blank">lkrispen@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hmm, the symbols are there now, but all threads are idle, DS is just
waiting on work to do. Which client do you expect to connect to DS,
maybe you need to debug this client.<div><div class="h5"><br>
<br>
<div>On 11/13/2014 11:02 AM, Walter van
Lille wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks Rich, I have installed the packages and run
gdb again. Hopefully the attached file is more useful.</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Nov 12, 2014 at 4:16 PM, Rich
Megginson <span dir="ltr"><<a href="mailto:rmeggins@redhat.com" target="_blank">rmeggins@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span>
<div>On 11/12/2014 05:42 AM, Walter van Lille wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks again for the assistance guys. I
have saved two files and included it here. Hope it
tells you more than it does me.</div>
</blockquote>
<br>
</span> These stack traces contain no useful symbols.<br>
<br>
Please read <a href="http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes" target="_blank">http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes</a>
to find out how to install the correct debuginfo packages
on your system. For IPA you will also need debuginfo
packages for ipa and slapi-nis<br>
<br>
If debuginfo-install is not working, see <a href="https://www.centos.org/forums/viewtopic.php?t=1710" target="_blank">https://www.centos.org/forums/viewtopic.php?t=1710</a><br>
<br>
If you simply cannot figure out how to install the
debuginfo packages, please email me with the following
information:<br>
<br>
$ rpm -q ipa-server slapi-nis 389-ds-base openldap db4 nss
nspr glibc<br>
<br>
and I will make them available to you<br>
<br>
NOTE: With debuginfo packages, the version of the
debuginfo package _must exactly match_ the version of the
associated package e.g. for <span>389-ds-base.x86_64
1.2.11.15-34.el6_5 you must have </span><br>
<span>389-ds-base-debuginfo.x86_64 1.2.11.15-34.el6_5<br>
<br>
If the versions do not match exactly, you will not get
the symbols.<br>
<br>
</span>
<div>
<div>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Wallter</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Nov 11, 2014 at
8:03 PM, Rich Megginson <span dir="ltr"><<a href="mailto:rmeggins@redhat.com" target="_blank">rmeggins@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span>
<div>On 11/11/2014 10:37 AM, Martin Basti
wrote:<br>
</div>
<blockquote type="cite">
<div>On 11/11/14 15:58, Rich Megginson
wrote:<br>
</div>
<blockquote type="cite">
<div>On 11/11/2014 06:20 AM, Ludwig
Krispenz wrote:<br>
</div>
<blockquote type="cite"> <br>
<div>On 11/11/2014 02:14 PM, Martin
Basti wrote:<br>
</div>
<blockquote type="cite">
<div>Ludiwg (CCed) this seems like
old (fixed?) DS bug.<br>
</div>
</blockquote>
hmm, it says limit is 2097152, so it
already has the new setting, but the
error message says the packet is
800MB<b><br>
</b></blockquote>
<br>
<b>Right. That usually means the
server was expecting an encrypted
SASL buffer from the client, but
instead the client thinks SASL
encryption negotiation failed and
just sent a plain LDAP buffer. What
version of 389-ds-base are you
using? rpm -q 389-ds-base<br>
<br>
<a href="https://fedorahosted.org/389/ticket/47416" target="_blank">https://fedorahosted.org/389/ticket/47416</a><br>
<br>
So, DO NOT increase your sasl io
buffer size - it will not fix the
problem, and it will leave you open
to DoS attacks.<br>
</b></blockquote>
<br>
He is using <br>
<div><br>
</div>
<div>CentOS release 6.5 (Final)<br>
</div>
<div>389-ds-base.x86_64
1.2.11.15-34.el6_5</div>
</blockquote>
<br>
<br>
</span> Ignore the "SASL encrypted packet
length exceeds maximum allowed limit"
error. All it means is the client has some
error and is canceling the connection.<br>
<br>
The bugzilla associated with <b><a href="https://fedorahosted.org/389/ticket/47416" target="_blank">47416</a> is targeted
for RHEL 7.1. The problem is that we were
never able to figure out what error the
client was getting that caused the client
to stop establishing the </b>SASL
encryption, and the original customer who
reported the problem dropped the case -
everything just started working, no further
errors. Note that 47416 doesn't really fix
the problem or address the root cause - the
root cause is some error on the client side
that causes it to stop doing encrypted SASL
I/O and send an LDAP unbind request.<br>
<br>
Let's get back to the actual problem - what
is the actual problem? The ldap server is
hanging or unresponsive? If so, then see <a href="http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs" target="_blank">http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs</a>.
Let's get some dirsrv stack traces during
the period of time when it appears to be
unresponsive.
<div>
<div><br>
<br>
<blockquote type="cite"> <br>
<blockquote type="cite"><b> <br>
</b>
<blockquote type="cite"><b> </b>
<blockquote type="cite">
<div> <br>
On 11/11/14 13:13, Walter van
Lille wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>I've just cleaned out
a ton of slapd_poll
timed out messages from
the output and changed
the names to protect the
innocent, :-)</div>
<div>Here is the output as
requested:</div>
<div><br>
</div>
<div><br>
</div>
<div><b>[05/Nov/2014:11:44:05
+0200] - SASL
encrypted packet
length exceeds maximum
allowed limit
(length=805565,
limit=2097152).
Change the
nsslapd-maxsasliosize
attribute in cn=config
to increase limit.</b></div>
</div>
<div><b><br>
</b></div>
<div><b>[10/Nov/2014:14:45:19
+0200] - slapd_poll(115)
timed out</b></div>
<div><b>[10/Nov/2014:14:45:19
+0200] sasl_io_enable -
Cannot enable SASL
security on connection
in CLOSING state</b></div>
<div><b>[10/Nov/2014:14:45:19
+0200] - Error: could
not add/remove IO layers
from connection</b></div>
<div>
<div><b>[11/Nov/2014:11:48:09
+0200] - slapd
shutting down -
signaling operation
threads</b></div>
<div><b>[11/Nov/2014:11:48:09
+0200] - slapd
shutting down -
waiting for 30 threads
to terminate</b></div>
</div>
<div>
<div><b>[11/Nov/2014:13:14:12
+0200] - slapd
shutting down -
closing down internal
subsystems and plugins</b></div>
<div><b>[11/Nov/2014:13:14:12
+0200] - Waiting for 4
database threads to
stop</b></div>
<div><b>[11/Nov/2014:13:14:13
+0200] - All database
threads now stopped</b></div>
<div><b>[11/Nov/2014:13:14:13
+0200] - slapd
stopped.</b></div>
<div><b>[11/Nov/2014:13:26:35
+0200] -
389-Directory/<a href="http://1.2.11.15" target="_blank">1.2.11.15</a> B2014.219.179
starting up</b></div>
<div><b>[11/Nov/2014:13:26:35
+0200]
schema-compat-plugin -
warning: no entries
set up under
cn=computers,
cn=compat,dc=sample,dc=example</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Skipping CoS
Definition cn=Password
Policy,cn=accounts,dc=sample,dc=example--no
CoS Templates found,
which should be added
before the CoS
Definition.</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Skipping CoS
Definition cn=Password
Policy,cn=accounts,dc=sample,dc=example--no
CoS Templates found,
which should be added
before the CoS
Definition.</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - slapd
started. Listening on
All Interfaces port
389 for LDAP requests</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Listening on
All Interfaces port
636 for LDAPS requests</b></div>
<div><b>[11/Nov/2014:13:26:36
+0200] - Listening on
/var/run/slapd-SAMPLE-EXAMPLE.socket
for LDAPI requests</b></div>
<div><b>[11/Nov/2014:13:57:08
+0200] -
slapd_poll(78) timed
out</b></div>
</div>
<div><b><br>
</b></div>
<div><b><br>
</b></div>
<div><b><br>
</b></div>
<div><br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Tue, Nov 11, 2014 at
1:19 PM, Martin Basti <span dir="ltr"><<a href="mailto:mbasti@redhat.com" target="_blank">mbasti@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>IMHO It's DS
bug, can you share
DS error log?<br>
pspacek CCed to
examine named
logs.<br>
<br>
Martin^2
<div>
<div><br>
<br>
On 11/11/14
12:13, Walter
van Lille
wrote:<br>
</div>
</div>
</div>
<div>
<div>
<blockquote type="cite">
<div dir="ltr">Hi
Martin, thanks
for the reply.
<div>My
version:
bind-dyndb-ldap-2.3-5.el6.x86_64</div>
<div>The
server doesn't
have
journalctl
installed but
I have the
outputs from
the messages
and named.run
files that I
included here:</div>
<div><br>
</div>
<div>Messages:</div>
<div><br>
</div>
<div>
<div><b>Nov 11
12:30:13
freeipa
named[1481]:
error (network
unreachable)
resolving
'example.example.com.10.123.123.123/A/IN':
2001:500:2f::f#53</b></div>
<div><b>Nov 11
12:30:23
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
<div><b>Nov 11
12:30:23
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
<div><b>Nov 11
12:30:33
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
<div><b>Nov 11
12:30:33
freeipa
named[1481]:
LDAP query
timed out. Try
to adjust
"timeout"
parameter</b></div>
</div>
<div><br>
</div>
<div>Named.run:</div>
<div><br>
</div>
<div>
<div><b>client
10.123.123.123#42639:
transfer of
'example.example/IN':
AXFR-style
IXFR started</b></div>
<div><b>client
10.123.123.123#42639:
transfer of
''example.example/IN':
AXFR-style
IXFR ended</b></div>
<div><b>client
10.123.123.123#46912:
transfer of
'10.123.123.123.in-addr.arpa/IN':
AXFR-style
IXFR started</b></div>
<div><b>client
10.123.123.123#46912:
transfer of
'10.123.123.123.in-addr.arpa/IN':
AXFR-style
IXFR ended</b></div>
<div><b>LDAP
query timed
out. Try to
adjust
"timeout"
parameter</b></div>
<div><b>LDAP
query timed
out. Try to
adjust
"timeout"
parameter</b></div>
<div><b>LDAP
query timed
out. Try to
adjust
"timeout"
parameter</b></div>
</div>
<div><br>
</div>
<div>I just
replaced the
IPs and the
actual names
with something
more generic.</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Walter</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Thu, Nov 6,
2014 at 5:00
PM, Martin
Basti <span dir="ltr"><<a href="mailto:mbasti@redhat.com" target="_blank">mbasti@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div>
<div>On
06/11/14
14:58, Walter
van Lille
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>I need
some
assistance
please.</div>
<div>I've
taken over an
IPA server to
manage a few
months ago,
and it was
working fine
until recently
when it
started acting
up seemingly
off its own
accord.</div>
<div>When I do
an ipactl
status it
basically
gives an
output as
shown below:</div>
<div><br>
</div>
<div><br>
</div>
<div><b>Directory
Service:
RUNNING<br>
</b></div>
<div><b><br>
</b></div>
<div>
<div><b>Loooooooooooooooooooooooooooooooooooooooooooooooooong
pause... (To
the tune of 7
minutes
sometimes)</b></div>
</div>
<div><b><br>
</b></div>
<div>
<div><b>KDC
Service:
RUNNING</b></div>
<div><b>KPASSWD
Service:
RUNNING</b></div>
<div><b>DNS
Service:
RUNNING</b></div>
<div><b>MEMCACHE
Service:
RUNNING</b></div>
<div><b>HTTP
Service:
RUNNING</b></div>
<div><b>CA
Service:
RUNNING</b></div>
<div><b>ADTRUST
Service:
RUNNING</b></div>
<div><b>EXTID
Service:
RUNNING</b></div>
</div>
<div><br>
</div>
<div>Running
top showed
that ns-slapd
was munching
almost all my
resources, but
I got that
fixed by
upping the
cache.
Unfortunately
this did not
correct the
issue and it
still reacts
in the same
fashion,
although the
resources have
been freed up
now.</div>
<div>I've
noticed that
when I run dig
on either the
local server
or a remote
machine that
the query
basically just
times out as
shown here:</div>
<div><br>
</div>
<div>
<div> <b>dig
freeipa.myexample.sample</b></div>
<div><b><br>
</b></div>
<div><b>;
<<>>
DiG
9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1
<<>>
freeipa.myexample.sample</b></div>
<div><b>;;
global
options: +cmd</b></div>
<div><b>;;
connection
timed out; no
servers could
be reached</b></div>
</div>
<div><br>
</div>
<div>When the
KDC service
fails to
start, then
name lookups
seem OK, but
authentication
fails.
otherwise it's
dead in the
water.</div>
<div><br>
</div>
<div>This also
happens:</div>
<div>
<div><br>
</div>
<div><b>sudo
ipactl status</b></div>
<div><b>Directory
Service:
RUNNING</b></div>
<div><b>Unknown
error when
retrieving
list of
services from
LDAP:</b></div>
</div>
<div><b><br>
</b></div>
<div>My
software setup
is as follows:</div>
<div><br>
</div>
<div><b>CentOS
release 6.5
(Final)<br>
</b></div>
<div><b>389-ds-base.x86_64
1.2.11.15-34.el6_5<br>
</b></div>
<div><b>bind.x86_64
32:9.8.2-0.23.rc1.el6_5.1<br>
</b></div>
<div>
<div><b>bind-dyndb-ldap.x86_64</b></div>
<div><b>bind-libs.x86_64
32:9.8.2-0.23.rc1.el6_5.1</b></div>
<div><b>bind-utils.x86_64
32:9.8.2-0.23.rc1.el6_5.1</b></div>
<div><b>rpcbind.x86_64
0.2.0-11.el6
@anaconda-CentOS-201311291202.x86_64/6.5</b></div>
<div><b>samba4-winbind.x86_64</b></div>
</div>
<div><b>krb5-server.x86_64
1.10.3-15.el6_5.1<br>
</b></div>
<div><b><br>
</b></div>
<div><b>Linux
2.6.32-431.29.2.el6.x86_64
#1 SMP Tue Sep
9 21:36:05 UTC
2014 x86_64
x86_64 x86_64
GNU/Linux<br>
</b></div>
<div><br>
</div>
<div>It's not
a permanent
situation as
it sometimes
runs 100% for
a while, but
80% of the
time it is
unusable. If
anybody can
assist me,
please be so
kind.</div>
<div><br>
</div>
<div>Regards,</div>
<div><br>
</div>
<div>Walter</div>
<div><br>
</div>
</div>
</blockquote>
</div>
</div>
Hello please
which version
of
bind-dyndb-ldap
do you use?<br>
I had similar
issue with
bind-dyndb-ldap,
but it was
development
version, I'm
not sure if
this is your
case.<br>
When named was
failing,
dirserv was
really slow.<br>
<br>
Can you send
journalctl -b
-u named log
when dig
doesn't work??<span><font color="#888888"><br>
<br>
<pre cols="72">--
Martin Basti</pre>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<br>
</div>
</div>
<span><font color="#888888">
<pre cols="72">--
Martin Basti</pre>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<br>
<pre cols="72">--
Martin Basti</pre>
</blockquote>
<br>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
<br>
<pre cols="72">--
Martin Basti</pre>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
</div>
</div>
</div>
<br>
--<br>
Manage your subscription for the Freeipa-users
mailing list:<br>
<a href="https://www.redhat.com/mailman/listinfo/freeipa-users" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a><br>
Go To <a href="http://freeipa.org" target="_blank">http://freeipa.org</a>
for more info on the project<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
</blockquote>
<br>
</div></div></div>
<br>--<br>
Manage your subscription for the Freeipa-users mailing list:<br>
<a href="https://www.redhat.com/mailman/listinfo/freeipa-users" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a><br>
Go To <a href="http://freeipa.org" target="_blank">http://freeipa.org</a> for more info on the project<br></blockquote></div><br></div>