[Libguestfs] alternatives for hooking dlopen() without LD_LIBRARY_PATH or LD_AUDIT?

[Carlos O'Donell]

On 2/14/20 4:29 PM, Eric Blake wrote:
> On 2/14/20 1:02 PM, Eric Blake wrote:
> 
>> Writing my own dlopen() wrapper directly in nbdkit seems like a
>> non-starter (my override has to come from a shared library before
>> it can replace the shared version that would be imported from -ldl,
>> at least for all subsequent shared library loads that want to bind
>> to the override).
> 
> Maybe I spoke too soon. I've tried another approach that looks like
> it will do what I want: put my shim dlopen() in a shared library, but
> link nbdkit against that shared library PRIOR to -ldl (so that name
> lookup for dlopen resolves there first).  The shim library in turn
> depends on -ldl so that dlsym(RTLD_NEXT, "dlopen") still lets me get
> to the real dlopen.  And by linking it directly into nbdkit, rather
> than into the nbdkit-vddk-plugin.so that gets loaded later, the first
> bound dlopen() in use for all subsequent loads is from my shim.  It's
> still a bit less clean than I'd like (it requires tighter coupling
> between nbdkit and nbdkit-vddk-plugin.so than what used to exist),
> but the fact that it works without dlmopen() or LD_LIBRARY_PATH is in
> its favor.  I'm now polishing up the experiment, and will post the
> patch when it's ready.

I think that's the best solution you're going to get.

The alternatives (LD_LIBRARY_PATH, direct loader invocation, dlmopen)
all have limitations that aren't helpful to your particular design.

You have design strategies that look like this:

- Move the object higher in the search order at link time (interposition)
  - Investigate static link order.
  - Investigate dynamic loader search order

- Change what object is searched for
  - LD_LIBRARY_PATH, DT_RPATH, DT_RUNPATH, etc.
  - LD_AUDIT with la_objsearch()

Your "shim dlopen()" is a case of moving the static link order
around to ensure your shim is used first.

-- 
Cheers,
Carlos.