[libvirt] [PATCH rust 1/5] fix bug integration test with rust multithreading

Sahid Orentino Ferdjaoui sahid.ferdjaoui at canonical.com
Fri Jul 5 10:40:56 UTC 2019


On Fri, Jul 05, 2019 at 09:47:14AM +0100, Daniel P. Berrangé wrote:
> On Thu, Jul 04, 2019 at 12:15:25PM +0200, Sahid Orentino Ferdjaoui wrote:
> > The open_auth test cases were randomly failling. It seems that because
> > tests are executed in parallel if a sasl connection is open when an
> > other happen in same time from the same libvirt connection an error
> > happens: "Failed to start SASL negotiation: -4 (SASL(-4): no mechanism
> > available: No worthy mechs found)".
> 
> This doesn't really make sense. Libvirt itself is fully multi-threaded
> so it should be fine to be opening connections in parallel.
> 
> If there's a non-deterministic failure happening this suggests a race
> condition bug somewhere, which shouldn't just be ignored by changing
> the tests to avoid hitting the race,

You are completely right I did lot of tests and I was not able to find
the root cause. I was only able to reproduce the problem when I'm
using the test framework and even with that if I'm using
test-threads=1, no problems, or if I'm executing only one of them
several times, no problems. Even, running 2 of example/auth.rs in
parallel lot of time does not cause any issues.

That is the leak which sometime happens:

==20734==
==20734== HEAP SUMMARY:
==20734==     in use at exit: 165,843 bytes in 852 blocks
==20734==   total heap usage: 4,141 allocs, 3,289 frees, 1,923,649 bytes allocated
==20734==
==20734== 544 bytes in 1 blocks are definitely lost in loss record 103 of 121
==20734==    at 0x4C2CE8E: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20734==    by 0xA780E5E: _plug_buf_alloc (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734==    by 0xA77A53D: ??? (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734==    by 0xA77F017: ??? (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734==    by 0xA77F8B0: ??? (in /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25)
==20734==    by 0x7DF85EF: sasl_client_step (in /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25)
==20734==    by 0x7DF896D: sasl_client_start (in /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25)
==20734==    by 0x4FF0756: virNetSASLSessionClientStart (in /usr/lib/libvirt.so.0.1002.2)
==20734==    by 0x4FD1AD4: ??? (in /usr/lib/libvirt.so.0.1002.2)
==20734==    by 0x4FD27E5: ??? (in /usr/lib/libvirt.so.0.1002.2)
==20734==    by 0x4F6C4E5: ??? (in /usr/lib/libvirt.so.0.1002.2)
==20734==    by 0x4F6F00C: virConnectOpenAuth (in /usr/lib/libvirt.so.0.1002.2)
==20734==
==20734== LEAK SUMMARY:
==20734==    definitely lost: 544 bytes in 1 blocks
==20734==    indirectly lost: 0 bytes in 0 blocks
==20734==      possibly lost: 0 bytes in 0 blocks
==20734==    still reachable: 165,299 bytes in 851 blocks
==20734==         suppressed: 0 bytes in 0 blocks
==20734== Reachable blocks (those to which a pointer was found) are not shown.
==20734== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==20734==
==20734== For counts of detected and suppressed errors, rerun with: -v
==20734== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

I also have no clues why the heap is not completely freed. I will try
to investigate a bit more.

Thanks,
s.

> > 
> > Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui at canonical.com>
> > ---
> >  tests/integration_qemu.rs | 15 +++++++++++----
> >  1 file changed, 11 insertions(+), 4 deletions(-)
> > 
> > diff --git a/tests/integration_qemu.rs b/tests/integration_qemu.rs
> > index 49e07c4..7db43d5 100644
> > --- a/tests/integration_qemu.rs
> > +++ b/tests/integration_qemu.rs
> > @@ -89,6 +89,16 @@ fn test_create_storage_pool_and_vols() {
> >  #[test]
> >  #[ignore]
> >  fn test_connection_with_auth() {
> > +    // Rust is excecuting tests in parallel (threads), if a sasl
> > +    // connection is open when an other happen in same time from the
> > +    // same libvirt connection an error happens: "Failed to start SASL
> > +    // negotiation: -4 (SASL(-4): no mechanism available: No worthy
> > +    // mechs found)".
> > +    connection_with_auth_ok();
> > +    connection_with_auth_wrong();
> > +}
> > +
> > +fn connection_with_auth_ok() {
> >      fn callback(creds: &mut Vec<ConnectCredential>) {
> >          for cred in creds {
> >              match cred.typed {
> > @@ -118,10 +128,7 @@ fn test_connection_with_auth() {
> >      }
> >  }
> >  
> > -
> > -#[test]
> > -#[ignore]
> > -fn test_connection_with_auth_wrong() {
> > +fn connection_with_auth_wrong() {
> >      fn callback(creds: &mut Vec<ConnectCredential>) {
> >          for cred in creds {
> >              match cred.typed {
> > -- 
> > 2.20.1
> > 
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|




More information about the libvir-list mailing list