[PATCH] add build dependency on lxc_protocol.h to remote_daemon

[Daniel P. Berrangé]

On Wed, Mar 16, 2022 at 09:39:52AM +0100, Michal Prívozník wrote:
> On 3/14/22 15:58, Slater, Joseph wrote:
> > I was really only wanting to make sure that both remote_driver and remote_daemon have access to lxc_protocol.h.  I am not an expert in meson builds but I think the only way to do that is have a loop outside of remote_daemon_generated and remote_driver_generated.  You could start with an empty remote_daemon_generated and append remote_xxx_generated after the foreach loop instead of initializing the way I did.
> > 
> > So, your 1) is just a side-effect, not something I was explicitly seeking.
> > 
> 
> Hey, I'm no meson expert either, but one thing that I've noticed is that
> with this patch I'm seeing more tasks that meson wants to run:
> 
> Without the patch: 1410
> With the patch: 1447

While likely correct in this case, be wary of believing what Meson
prints as the total number of tasks it intends to run upfront. I've
seen cases where it claimed it needed to run 100's of tasks, then
after running 3 tasks, it changed it mind and decided it only
needed 15 tasks :-)

> My understanding was that we need only to order tasks not add new ones.
> Now, looking at generated build.ninja I can see why:
> 
> libvirt.git $ grep "^build.*lxc_protocol.c.o:" build/build.ninja | cut
> -d':' -f1
> build
> src/remote/libvirt_remote_driver.a.p/meson-generated_.._lxc_protocol.c.o
> build src/libvirtd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtproxyd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtinterfaced.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtnetworkd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtnodedevd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtnwfilterd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtsecretd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtstoraged.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtxend.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtlxcd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtchd.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtqemud.p/meson-generated_.._remote_lxc_protocol.c.o
> build src/virtvboxd.p/meson-generated_.._remote_lxc_protocol.c.o
> 
> Meson decided it's faster to compile remote_lxc_protocol.c multiple
> times, for different daemons, each time with the same set of arguments
> (not shown in my paste above), than to compile it just once. I'm sure
> virqemud is going to be happy with understanding LXC driver protocol,
> but I am not happy about that. Le sigh.

I wouldn't say it has decided it is faster to compile many times.

In the past meson wasn't compiling it at all for the daemons.

AFAICT, currently the daemons only get access to a the
remote_protocol.c.o implementations indirectly through
linking to libvirt.so

With this patch, we are explicitly telling meson to link
the daemons with remote_protocol.c.o.

Meson strives for maximum correctness, so if an object file
is used from many output targets, it will compile the source
file separately for each target, to take into account differences
in compiler flags used for each output target.

IOW, meson is doing exactly what we told it to do here.

The result is that the daemons actually get 2 copies of the
the remote_protocol.c.o - one copy is directly linked into
the binary, and one copy is indirectly via libvirt.so

> Pavel, since you introduced meson, can you please shed more light into
> here? How should we set up ORDERING and not DEPENDENCY? In Makefile it's
> as easy as:

ORDERING / DEPENDENCY are the same thing. This patch is not actually
setting up a dependancy - it is explicitly telling meson to compile the
files many times.

I suspect what we need is this:

diff --git a/src/remote/meson.build b/src/remote/meson.build
index b2aafe6320..769fc2e97f 100644
--- a/src/remote/meson.build
+++ b/src/remote/meson.build
@@ -3,7 +3,8 @@ remote_driver_sources = [
   'remote_sockets.c',
 ]
 
-remote_driver_generated = []
+remote_generated_sources = []
+remote_generated_headers = []
 
 foreach name : [ 'remote', 'qemu', 'lxc' ]
   client_bodies_h = '@0 at _client_bodies.h'.format(name)
@@ -11,7 +12,7 @@ foreach name : [ 'remote', 'qemu', 'lxc' ]
   protocol_h = '@0 at _protocol.h'.format(name)
   protocol_x = '@0 at _protocol.x'.format(name)
 
-  remote_driver_generated += custom_target(
+  remote_generated_headers += custom_target(
     client_bodies_h,
     input: protocol_x,
     output: client_bodies_h,
@@ -21,7 +22,7 @@ foreach name : [ 'remote', 'qemu', 'lxc' ]
     capture: true,
   )
 
-  remote_driver_generated += custom_target(
+  remote_generated_sources += custom_target(
     protocol_h,
     input: protocol_x,
     output: protocol_h,
@@ -30,7 +31,7 @@ foreach name : [ 'remote', 'qemu', 'lxc' ]
     ],
   )
 
-  remote_driver_generated += custom_target(
+  remote_generated_sources += custom_target(
     protocol_c,
     input: protocol_x,
     output: protocol_c,
@@ -49,7 +50,9 @@ remote_daemon_sources = files(
   'remote_daemon_stream.c',
 )
 
-remote_daemon_generated = []
+remote_driver_generated = remote_generated_headers + remote_generated_sources
+# Daemon gets the impls indirectly via libvirt.so
+remote_daemon_generated = remote_generated_headers
 
 virt_ssh_helper_sources = files(
   'remote_sockets.c',

(not tested)

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|