[Freeipa-devel] [PATCH 0291-0294] Fix locking to prevent crashes and deadlocks

[Petr Spacek]

Hello,

I was fighting with random crashes for couple of days ... and discovered that 
run_exclusive_enter()/isc_task_beginexclusive() usage was completely incorrect 
and didn't actually lock anything.

This series of patches reworks internal locking (and related event system) to 
work around limitations of isc_task_beginexclusive() mechanism.

It would be better to get rid of isc_task_beginexclusive() completely but IMHO 
it is not possible because of BIND's dns_view*() functions have to be guarded 
with it.


Testing is going to be interesting because we are speaking about race conditions.

I used ~ 100 DNS zones, each zone had ~ 100 random domain names inside with 
random A/AAAA/TXT RRs. My LDIF is here:
http://people.redhat.com/~pspacek/a/2014/09/11/dns-test.ldif.xz

I was able to randomly reproduce various crashes when BIND was running with 
more threads than usually.

You can try to run BIND with this command (as root) and play games with -n 
parameter:
$ export KRB5_KTNAME="/etc/named.keytab"
$ named -4 -g -u named -m record -n 10

Please test also the case where BIND receives SIGINT during start-up. It is 
possible to run BIND with commands above and wait for message:
11-Sep-2014 11:54:58.092 running

At this point send SIGINT (CTRL+C) to BIND and see what happens. It could 
crash or deadlock.

It is necessary to send the signal before BIND prints this message:
11-Sep-2014 11:55:11.707 zone z1.test/IN: loaded serial 1410429304

Let me know if you need any assistance.

-- 
Petr^2 Spacek
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bind-dyndb-ldap-pspacek-0291-Rework-locking-in-settings.c-module.patch
Type: text/x-patch
Size: 13083 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140911/ad99cb88/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bind-dyndb-ldap-pspacek-0292-Rework-locking-around-DNS-views.patch
Type: text/x-patch
Size: 18372 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140911/ad99cb88/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bind-dyndb-ldap-pspacek-0293-Lock-syncrepl-queue-avoid-race-between-zones-and-res.patch
Type: text/x-patch
Size: 4052 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140911/ad99cb88/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bind-dyndb-ldap-pspacek-0294-Fix-deadlocks-on-shutdown-caused-by-sync_-_wait-call.patch
Type: text/x-patch
Size: 12826 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140911/ad99cb88/attachment-0003.bin>