Re: condvar performance in .59 vs .60

["dada1" <dada1 cosmosbay com>]

Hi Boris

The problem is that v.59 condvar implementation had a deadlock condition
that could be hit on SMP
I did had such lockups in a real application.

v.60 maybe slower, but it performs well (no deadlock).
The reason of the perf drop is that a context switch is done when a thread
calls pthread_cond_signal(), the awaken thread then have to block because of
the held condvar lock.

v.59 used to use a special futex op, FUTEX_REQUEUE, but it seems a potential
bug exists with this call.

If you need performance, you can code a special condvar implementation,
using futex calls.

You can indeed have a special case if you know that a condvar is always used
with the same user mutex, and if the condvar functions are always called
with this mutex held : No need to use another mutex inside the condvar to
protect the condvars state.

Eric Dumazet

> Good day,
>
> I was running some customary tests with different threading
> libraries and noticed significant performance drop once
> switched from nptl v.59 to v.60. Thought you might be interested.
>
> Test consists of three threads and two queues. One thread is
> consumer  and is extracting messages from both queues as fast as
> it can. The other two threads are suppliers (thread per queue) and
> insert messages into the queue as fast as they can.
>
> I have two sub-configurations of this test:
>
> The first is using one pthread_mutex/pthread_cond to protect/signal
> both queues.
>
> The second is using two pthread_mutex'es one for each queue plus
> one twomutex_cond conditional variable which is capable of handling
> two mutexes instead of one (the pseudo-code is provided at the end
> of the posting).
>
> Each supplier is inserting 1M messages (simply int) into a queue.
> Each queue is preallocated before the threads are run.
>
> The test is executed on dual P4 Xeon 2Ghz 1GB running 2.6.0-test5.
>
> And here is the result (nptl-0.59 vs nptl-0.60 vs LinuxThreads):
>
> Legend: rate is in messages per wakeup (consumer)
>         time is in sec:mlsec (so 1:8 means 1sec 800mlsec)
> _____________________________________________
>          |                 |                 |
>          |  pthread_cond   |  twomutex_cond  |
> library  |-----------------|-----------------|
>          |  rate  |  time  |  rate  |  time  |
> _________|________|________|________|________|
>          |        |        |        |        |
> nptl-59  |  3.5   |  1:8   |  6.2   |  1:8   |
> _________|________|________|________|________|
>          |        |        |        |        |
> nptl-60  |  1.5   | 19:48  | 13.2   |  2:2   |
> _________|________|________|________|________|
>          |        |        |        |        |
> lt       |  1.6   | 18:9   | 13.6   |  2:2   |
> _________|________|________|________|________|
>
>
> First variation of the test runs about 10 times longer on 60
> compared to 59. At the same time decrease in rate is less than
> 3 times.
>
> hth,
> -boris
>
>
> #include <pthread.h>
>
> typedef struct
> {
>   unsigned long long int seq_;
>   pthread_mutex_t mutex_;
>   pthread_cond_t  cond_;
>
> } twomutex_cond_t;
>
> int
> twomutex_cond_wait (twomutex_cond_t *c,
>                     pthread_mutex_t *m1,
>                     pthread_mutex_t *m2)
> {
>   unsigned long long int before, after;
>
>   /* get sequence number before we start unlocking */
>   pthread_mutex_lock (&c->mutex_);
>   before = c->seq_;
>   pthread_mutex_unlock (&c->mutex_);
>
>   pthread_mutex_unlock (m2);
>
>   /* get sequence number after we unlocked all mutexes except the last */
>   pthread_mutex_lock (&c->mutex_);
>   after = c->seq_;
>   pthread_mutex_unlock (&c->mutex_);
>
>   /* check if somebody has signalled while we were busy unlocking */
>   if (after > before) return 0;
>
>   /* if not then go to bed */
>   pthread_cond_wait (&c->cond_, m1);
>
>   /* note, that m1 is automatically locked by cond_wait */
>   pthread_mutex_unlock (m2);
>
>   return 0;
> }
>
> /*
>   twomutex_cond_signal () assumes that the caller has one of
>   the m1 or m2 locked.
>
>  */
>
> int
> twomutex_cond_signal (twomutex_cond_t *c)
> {
>   /* advance sequence number */
>   pthread_mutex_lock (&c->mutex_);
>   ++c->seq_;
>   pthread_mutex_unlock (&c->mutex_);
>
>   pthread_cond_signal (&c->cond_);
>
>   return 0;
> }
>
>
> -- 
> Phil-list mailing list
> Phil-list redhat com
> https://www.redhat.com/mailman/listinfo/phil-list
>