From rmeggins at redhat.com Fri Dec 1 17:55:38 2006 From: rmeggins at redhat.com (Richard Megginson) Date: Fri, 01 Dec 2006 10:55:38 -0700 Subject: [Fedora-directory-devel] Please review: Bug 218076: Server autoconf build crashes when stopped/started very quickly Message-ID: <45706C9A.80307@redhat.com> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=218076 Bug(s) fixed: 218076 Bug Description: Server autoconf build crashes when stopped/started very quickly Reviewed by: ??? Files: see diff Branch: HEAD Fix Description: The problem was that 3 of the database threads were being started and stopped before the 4th had a chance to start. So the thread count would start at 3 and drop to 0, and the dblayer_pre_close code would think everything was fine. The 4th thread is the checkpoint thread which was doing a db_checkpoint operation before incrementing the thread count. For some reason, on x86_64 with the system provided libdb-4.2, the checkpoint operation was taking longer than it usually does with our locally built libdb-4.2, so this allowed the other 3 threads to stop and start before the checkpoint thread had a chance to increment the thread count. The solution is to make sure the incrementing of the thread count occurs as early as possible in the thread function, before any executable code that might take any time. This should ensure that all of the threads start up and increment the thread count before the shutdown occurs. The second part of the solution is that, according to wtc, the NSPR maintainer, the PR_Atomic functions should not be used as a semaphore like this. So, the code was rewritten to use locks and condition variables. The code is not performance critical, so adding locking should not have any impact on performance. In addition, the new code is much cleaner, more correct, and more obvious about what it's doing. Platforms tested: RHEL4 x86_64 Flag Day: no Doc impact: no https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142601&action=diff From voltaire at idirect.com Fri Dec 1 20:21:12 2006 From: voltaire at idirect.com (Jeffrey C. Rombough) Date: Fri, 1 Dec 2006 15:21:12 -0500 Subject: [Fedora-directory-devel] unsubscribe In-Reply-To: <45706C9A.80307@redhat.com> Message-ID: <022401c71586$51f1f1c0$640a0a0a@andromeda> unsubscribe -----Original Message----- From: fedora-directory-devel-bounces at redhat.com [mailto:fedora-directory-devel-bounces at redhat.com] On Behalf Of Richard Megginson Sent: December 1, 2006 12:56 PM To: Fedora Directory server developer discussion. Subject: [Fedora-directory-devel] Please review: Bug 218076: Server autoconf build crashes when stopped/started very quickly https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=218076 Bug(s) fixed: 218076 Bug Description: Server autoconf build crashes when stopped/started very quickly Reviewed by: ??? Files: see diff Branch: HEAD Fix Description: The problem was that 3 of the database threads were being started and stopped before the 4th had a chance to start. So the thread count would start at 3 and drop to 0, and the dblayer_pre_close code would think everything was fine. The 4th thread is the checkpoint thread which was doing a db_checkpoint operation before incrementing the thread count. For some reason, on x86_64 with the system provided libdb-4.2, the checkpoint operation was taking longer than it usually does with our locally built libdb-4.2, so this allowed the other 3 threads to stop and start before the checkpoint thread had a chance to increment the thread count. The solution is to make sure the incrementing of the thread count occurs as early as possible in the thread function, before any executable code that might take any time. This should ensure that all of the threads start up and increment the thread count before the shutdown occurs. The second part of the solution is that, according to wtc, the NSPR maintainer, the PR_Atomic functions should not be used as a semaphore like this. So, the code was rewritten to use locks and condition variables. The code is not performance critical, so adding locking should not have any impact on performance. In addition, the new code is much cleaner, more correct, and more obvious about what it's doing. Platforms tested: RHEL4 x86_64 Flag Day: no Doc impact: no https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142601&action=diff -- Fedora-directory-devel mailing list Fedora-directory-devel at redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-devel From nhosoi at redhat.com Wed Dec 6 03:24:34 2006 From: nhosoi at redhat.com (Noriko Hosoi) Date: Tue, 05 Dec 2006 19:24:34 -0800 Subject: [Fedora-directory-devel] Please review: [Bug 202843] referential integrity plugin does not stack with Class of Service appliance In-Reply-To: <200612060320.kB63KGA5012077@bugzilla.redhat.com> References: <200612060320.kB63KGA5012077@bugzilla.redhat.com> Message-ID: <457637F2.6040406@redhat.com> Summary: referential integrity plugin does not stack with Class of Service appliance https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=202843 ------- Additional Comments From nhosoi at redhat.com 2006-12-05 13:12 EST ------- Created an attachment (id=142873) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142873&action=view) bug analysis and discussion ------- Additional Comments From nhosoi at redhat.com 2006-12-05 22:05 EST ------- Created an attachment (id=142912) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142912&action=view) cvs diffs for reviews Files: plugins/cos/cos_cache.c plugins/referint/referint.c slapd/delete.c slapd/modify.c slapd/modrdn.c slapd/pblock.c slapd/plugin.c slapd/slap.h slapd/slapi-plugin.h ------- Additional Comments From nhosoi at redhat.com 2006-12-05 22:07 EST ------- Created an attachment (id=142913) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142913&action=view) Fix description This attachment shows just ordering the plugins was not enough for updating the CoS cache. It is a sort of codicil. ------- Additional Comments From nhosoi at redhat.com 2006-12-05 22:12 EST ------- Created an attachment (id=142914) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142914&action=view) Test result: CoS is called after Referential Integrity Plugin Attachment referred by the previous comment. Thanks, --noriko -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From nhosoi at redhat.com Thu Dec 7 02:21:58 2006 From: nhosoi at redhat.com (Noriko Hosoi) Date: Wed, 06 Dec 2006 18:21:58 -0800 Subject: [Fedora-directory-devel] Please review (revised): [Bug 202843] referential integrity plugin does not stack with Class of Service appliance In-Reply-To: <457637F2.6040406@redhat.com> References: <200612060320.kB63KGA5012077@bugzilla.redhat.com> <457637F2.6040406@redhat.com> Message-ID: <45777AC6.5010701@redhat.com> > Summary: referential integrity plugin does not stack with Class of > Service appliance > > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=202843 I fixed a bug described in Comment #8. This is the revised diffs: ------- Additional Comments From nhosoi at redhat.com 2006-12-06 18:09 EST ------- Created an attachment (id=143004) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=143004&action=view) cvs diffs (revised) Fixed a bug reported in the previous comment (comment #8) That is, added a code to delete the plugin if the plugin is already in the list before the plugin with "name". Thank you, in advance. --noriko > ------- Additional Comments From nhosoi at redhat.com 2006-12-05 13:12 > EST ------- > Created an attachment (id=142873) > --> > (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142873&action=view) > > bug analysis and discussion > > ------- Additional Comments From nhosoi at redhat.com 2006-12-05 22:05 > EST ------- > Created an attachment (id=142912) > --> > (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142912&action=view) > > cvs diffs for reviews > > Files: > plugins/cos/cos_cache.c > plugins/referint/referint.c > slapd/delete.c > slapd/modify.c > slapd/modrdn.c > slapd/pblock.c > slapd/plugin.c > slapd/slap.h > slapd/slapi-plugin.h > > ------- Additional Comments From nhosoi at redhat.com 2006-12-05 22:07 > EST ------- > Created an attachment (id=142913) > --> > (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142913&action=view) > > Fix description > > > This attachment shows just ordering the plugins was not enough for > updating the CoS cache. It is a sort of codicil. > ------- Additional Comments From nhosoi at redhat.com 2006-12-05 22:12 > EST ------- > Created an attachment (id=142914) > --> > (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=142914&action=view) > > Test result: CoS is called after Referential Integrity Plugin > Attachment referred by the previous comment. > > Thanks, > --noriko -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From nhosoi at redhat.com Thu Dec 7 18:56:04 2006 From: nhosoi at redhat.com (Noriko Hosoi) Date: Thu, 07 Dec 2006 10:56:04 -0800 Subject: [Fedora-directory-devel] Please review: [Bug 183222] Directory Server hangs when running VLV search and update operations simultaneously. In-Reply-To: <200612071847.kB7IlP8m007882@bugzilla.redhat.com> References: <200612071847.kB7IlP8m007882@bugzilla.redhat.com> Message-ID: <457863C4.2090303@redhat.com> Summary: Directory Server hangs when running VLV search and update operations simultaneously. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=183222 There was another deadlock case between delete and vlv search... ------- Additional Comments From nhosoi at redhat.com 2006-12-07 13:47 EST ------- Created an attachment (id=143078) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=143078&action=view) cvs diff vlv.c File: back-ldbm/vlv.c Problem description: There was another source of deadlock. vlv_build_candidate_list creates db cursor in it. The current code locks the vlvSearchList, calls vlv_build_candidate_list, then unlock it after the function returns. Creating db cursor should not be inside of the vlvSearchList lock. Changes: Before creating db cursor, unlock vlvSearchList. It should be safe since there is no chance to traverse the vlvSearchList. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: From nhosoi at redhat.com Wed Dec 13 23:41:45 2006 From: nhosoi at redhat.com (Noriko Hosoi) Date: Wed, 13 Dec 2006 15:41:45 -0800 Subject: [Fedora-directory-devel] Please review: [Bug 195305] make new_task() non-static & [Bug 195307] task registration by plugins is wiped by task_init() In-Reply-To: <200612132326.kBDNQWgZ010045@bugzilla.redhat.com> References: <200612132326.kBDNQWgZ010045@bugzilla.redhat.com> Message-ID: <45808FB9.5050505@redhat.com> Summary: make new_task() non-static https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=195305 Problem description: The current prototype for task.c:new_task() is: static Slapi_Task *new_task(const char *dn) Can you make it non-static? new_task() is used to register a new task request, eg cn=mytask_200606091405, cn=mytask, cn=tasks, cn=config. new_task() is a pretty simple function and it could be recreated by people writing custom tasks, except that to register a new task request one needs access to the global task list and the mutex around it. Currently new_task() is the only (indirect) access to those objects for code outside libslapd. So we need either a getter function for the mutex and global task list, or new_task() should be callable from plugin libraries. ------- Additional Comments From nhosoi at redhat.com 2006-12-13 18:26 EST ------- Created an attachment (id=143579) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=143579&action=view) cvs diffs ( Thank you, Ulf, for your response! Here's the proposal... Files: slapi-plugin.h slapi-private.h mapping_tree.c task.c Change description: Expose new_task and destroy_task as slapi_new_task and slapi_destroy_task: +/* + * slapi_new_task: create new task, fill in DN, and setup modify callback + * argument: + * dn: task dn + * result: + * Success: Slapi_Task object + * Failure: NULL + */ +Slapi_Task *slapi_new_task(const char *dn); + +/* slapi_destroy_task: destroy a task + * argument: + * task: task to destroy + * result: + * none + */ +void slapi_destroy_task(void *arg); We also have to open the task status macros: +/* task states */ +#define SLAPI_TASK_SETUP 0 +#define SLAPI_TASK_RUNNING 1 +#define SLAPI_TASK_FINISHED 2 +#define SLAPI_TASK_CANCELLED 3 And task support functions: +int slapi_task_register_handler(const char *name, dseCallbackFn func); +void slapi_task_status_changed(Slapi_Task *task); +void slapi_task_log_status(Slapi_Task *task, char *format, ...) ------------------------------------------------------------------------ Summary: task registration by plugins is wiped by task_init() https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=195307 Problem description: main() calls task_init() after plugin_startall(). I think that order is necessary, except task_init() calls cleanup_old_tasks(). Therefore any task subtrees in the DSE that were added as part of task registration by plugins are deleted. As a workaround I'm finding some way to register my task at a later point, maybe with an eq delay or triggered from another common plugin callback like SLAPI_PLUGIN_PRE_SEARCH_FN maybe - I'm open to suggestions. But eventually, maybe it would be good to rework the way this works somehow. I think the community would dig being able to write plugins for custom tasks and it's a bit cumbersome and undocumented at the moment. ------- Additional Comments From nhosoi at redhat.com 2006-12-13 18:08 EST ------- Created an attachment (id=143572) --> (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=143572&action=view) cvs diffs (slapi-private.h, main.c, task.c) Files: slapi-private.h main.c task.c Change description: I agree with Nathan that we don't have "to have one central cleanup function to cleanup any type of task entry that may be hanging around" for the current task usage. I'm a little worried if we expose the interface and allow users to add task plugins, then we may have more chances to see such leftovers. So, cleaning up all the tasks belonging to cn=tasks,cn=config first could be more meaningful. I renamed the cleanup function cleanup_old_tasks to task_cleanup and made it available just for the server ns-slapd (it's declared in slapi-private.h) and call it before plugin_startall in main. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3237 bytes Desc: S/MIME Cryptographic Signature URL: