[libvirt] Entering freeze for libvirt-3.5.0

[Daniel P. Berrange]

On Tue, Jul 04, 2017 at 05:32:03PM +0100, Daniel P. Berrange wrote:
> On Tue, Jul 04, 2017 at 12:21:21PM +0100, Daniel P. Berrange wrote:
> > On Tue, Jul 04, 2017 at 01:03:52PM +0200, Andrea Bolognani wrote:
> > > On Sat, 2017-07-01 at 14:18 +0400, Roman Bogorodskiy wrote:
> > > > > qemuxml2argvtest fails consistently in my FreeBSD guest.
> > > > 
> > > > I guess that's caused by clang inlining functions that are mocked
> > > > (specifically, some numa related stuff); I think that was discussed
> > > > several times already. Anyway, it should work fine with '-O0' in CFLAGS.
> > > 
> > > Well, would you look at that. It does indeed work flawlessly
> > > when compiled without optimizations! :O
> > > 
> > > I'm not sure if that would be considered a reasonable
> > > compromise to get the test suite running on FreeBSD in the
> > > context of CI, though. I think it working reliably without
> > > messing with CFLAGS would be a requirement; others might
> > > disagree.
> > 
> > Hmm, I thought I fixed that problem when I introduce this patch:
> > 
> >   commit 728cacc8abed2b8de39e7b96fa42fde6850ec23a
> >   Author: Daniel P. Berrange <berrange at redhat.com>
> >   Date:   Fri Apr 7 15:07:49 2017 +0100
> > 
> >     annotate all mocked functions with noinline
> > 
> > 
> > This made us annotate all mocked functions with noinline, which was
> > sufficient to make CLang builds pass tests on Ubuntu VMs.
> 
> So I did some debugging and this is wierder than I can imagine possible.
> 
> I put a printf statement  in virNumaGetMaxNode in qemuxml2argvmock.c
> to print out the return value.
> 
> I also put a printf statement in virNumaNodeIsAvailable in virnuma.c
> (in the non-NUMACTL conditional block), that prints out the return
> value received from virNumaGetMaxNode
> 
> The first printf displays 7, while the second printf displays -1
> 
> So we're definitely calling our mock override, but the return
> value is getting mangled when seen by the caller, which is a
> giant wtf to me.

I wrote an isolated test case

$ cat lib.h

int get_max(void) __attribute__((noinline));

int is_ok(int i);



$ cat lib.c

#include <stdio.h>
#include "lib.h"

int get_max(void)
{
  fprintf(stderr, "In original max, returning 3\n");
  return 3;
}

int is_ok(int i)
{
  int max = get_max();
  fprintf(stderr, "Received max %d\n", max);
  return i > 0 && i <= max;
}



$ cat mock.c

#include <stdio.h>
#include "lib.h"


int get_max(void)
{
  fprintf(stderr, "In mock max, returning 7\n");
  return 7;
}



$ cat run.c

#include <stdio.h>
#include "lib.h"

int main(int argc, char **argv)

{
  fprintf(stderr, "Is 5 in range ? %d\n", is_ok(5));
}


$ clang -O2 -g -Wall -shared -o libdemo.so  -fPIC lib.c
$ clang -O2 -g -Wall -shared -o mock.so -fPIC  mock.c
$ clang -O2 -Wall -o run run.c -L. -ldemo


$ ./run
In original max, returning 3
Received max 3
Is 5 in range ? 0
$ LD_PRELOAD=mock.so  ./run
In mock max, returning 7
Received max 3
Is 5 in range ? 0


$ clang -O0 -g -Wall -shared -o libdemo.so  -fPIC lib.c
$ LD_PRELOAD=mock.so  ./run
In mock max, returning 7
Received max 7
Is 5 in range ? 1


So clang is definitely *not* inlining the function, *is* running out mock
function, but none the less getting the return value from the original
function.

Turning on optimizer debugging i see


$ clang -O2 -g -Wall -shared -o libdemo.so -Rpass=.* -fPIC lib.c
lib.c:7:3: remark: marked this call a tail call candidate [-Rpass=tailcallelim]
  fprintf(stderr, "In original max, returning 3\n");
  ^
lib.c:11:15: remark: marked this call a tail call candidate [-Rpass=tailcallelim]
int is_ok(int i)
              ^
lib.c:13:13: remark: marked this call a tail call candidate [-Rpass=tailcallelim]
  int max = get_max();
            ^
lib.c:13:7: remark: marked this call a tail call candidate [-Rpass=tailcallelim]
  int max = get_max();
      ^
lib.c:14:3: remark: marked this call a tail call candidate [-Rpass=tailcallelim]
  fprintf(stderr, "Received max %d\n", max);
  ^


so, I'm thinking this problem is a result of tail call optimization making
an assumption that is violated when mocking the function.

I'm unclear how to prevent tail call optimization without the big hammer
of -O0 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|