[Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container

[Nathan Kinder]

Hi,

I'm running into a strange problem related to ntpd when trying to use
IPA in a container.  I'm using the adelton/freeipa-server:fedora-21 and
adelton/freeipa-client:fedora-21 docker images.  Basically, the client
install hangs when it runs ntpd.  This is reproducible on two different
docker hosts of mine, so it will probably easily reproduce for others as
well.  Below are the steps I'm using.

Install IPA server in F21 container:

--------------------------------------------------------------------
[root at localhost ~]# docker run --name freeipa-server-container -d -h
ipa.example.test -e PASSWORD=Secret123 adelton/freeipa-server:fedora-21
875007ab561ff62ea45dde5e8a5e320a209c63b3c8fc52bd4ca7b22561d1bbf0
[root at localhost ~]# docker logs freeipa-server-container
...
FreeIPA server configured.
Go loop.
--------------------------------------------------------------------

Install IPA client in F21 container and link it to the IPA server
container.  This will hang indefinitely when it tries to run ntpd to
sync the time before getting the admin ticket:

--------------------------------------------------------------------
[root at localhost ~]# docker run --name client -h client.example.test
--link freeipa-server-container:ipa -e PASSWORD=Secret123 -e
IPA_CLIENT_INSTALL="--debug" -it adelton/freeipa-client:fedora-21
...
Synchronizing time with KDC...
Search DNS for SRV record of _ntp._udp.example.test
DNS record found: 0 100 123 ipa.example.test.
Starting external process
args='/usr/sbin/ntpd' '-qgc' '/tmp/tmpRhhyCz'
--------------------------------------------------------------------

If I use nsenter to go into the client container and kill ntpd, the
install continues and completes.  I also confirmed that the ntpd config
file that we create in /tmp is correct.  From within the client
container (via nsenter), running 'ntpd -qgc' with a conf file that
points to the IPA server just loops endlessly.

I looked into the IPA server container, and ntpd is not running.  The
ipaserver-install.log shows that it attempts to start (which returns 0),
but the service is not active afterwards:

--------------------------------------------------------------------
...
2015-01-14T22:57:02Z DEBUG   [4/4]: starting ntpd
2015-01-14T22:57:02Z DEBUG Starting external process
2015-01-14T22:57:02Z DEBUG args='/bin/systemctl' 'start' 'ntpd.service'
2015-01-14T22:57:03Z DEBUG Process finished, return code=0
2015-01-14T22:57:03Z DEBUG stdout=
2015-01-14T22:57:03Z DEBUG stderr=
2015-01-14T22:57:03Z DEBUG Starting external process
2015-01-14T22:57:03Z DEBUG args='/bin/systemctl' 'is-active' 'ntpd.service'
2015-01-14T22:57:04Z DEBUG Process finished, return code=3
2015-01-14T22:57:04Z DEBUG stdout=inactive

2015-01-14T22:57:04Z DEBUG stderr=
2015-01-14T22:57:04Z DEBUG   duration: 1 seconds
2015-01-14T22:57:04Z DEBUG Done configuring NTP daemon (ntpd).
...
--------------------------------------------------------------------

It seems that this causes ntpd on the F21 client to just loop endlessly
since it never sees a response.  We use ntpdate on F20, which bails out
and skips the time update on a F20 client when the server is unavailable:

--------------------------------------------------------------------
...
2015-01-15T03:29:11Z DEBUG args=/usr/sbin/ntpdate -U ntp -s -b -v
ipa.example.test
2015-01-15T03:29:11Z DEBUG Process finished, return code=1
2015-01-15T03:29:11Z DEBUG stdout=
2015-01-15T03:29:11Z DEBUG stderr=
2015-01-15T03:29:11Z DEBUG Starting external process
2015-01-15T03:29:11Z DEBUG args=/usr/sbin/ntpdate -U ntp -s -b -v
ipa.example.test
2015-01-15T03:29:11Z DEBUG Process finished, return code=1
2015-01-15T03:29:11Z DEBUG stdout=
2015-01-15T03:29:11Z DEBUG stderr=
2015-01-15T03:29:11Z WARNING Unable to sync time with IPA NTP server,
assuming the time is in sync. Please check that 123 UDP port is opened.
...
--------------------------------------------------------------------

I can do a 'systemctl start ntpd.service' on the IPA server container,
and it does start up successfully.  It never seems to automatically
start though, even if I restart the IPA server docker container.  I did
confirm that ntpd.service is enabled with systemctl, yet it doesn't
start automatically.

The /sbin/ipa-server-configure-first entrypoint script for the server
image does a 'systemctl start-enabled' to bring up all of the services,
which results in this output in /var/log/systemctl.log:

--------------------------------------------------------------------
[start-enabled]
[start ntpd.service]
Running [export OPTIONS="-g -x"; /usr/sbin/ntpd -u ntp:ntp $OPTIONS]
Marked pid [15] for [ntpd.service]
Marked process name [/usr/sbin/ntpd] for [ntpd.service]
...
--------------------------------------------------------------------

This is the same log output that is generated if I manually run
'systemctl start ntpd.service' from within the container, but the ntpd
process stays around when I start it this way.  It's hard to tell what
might be happening to ntpd, as there is no journal in the container.

I'm continuing to debug this, but I thought I'd share my findings thus
far in case anyone else has seen this or has any ideas for tracking the
problem down.  Any ideas?

Thanks,
-NGK