[libvirt] [PATCH 4/7] libvirt: support an "embed" URI path selector for opening drivers
Cole Robinson
crobinso at redhat.com
Tue Dec 17 17:41:27 UTC 2019
On 12/2/19 10:03 AM, Daniel P. Berrangé wrote:
> The driver URI scheme:
>
> "$drivername:///embed?root=/some/path"
>
> enables a new way to use the drivers by embedding them directly in the
> calling process. To use this the process must have a thread running the
> libvirt event loop. This URI will then cause libvirt to dynamically load
> the driver module and call its global initialization function. This
> syntax is applicable to any driver, but only those will have been
> modified to support a custom root directory and embed URI path will
> successfully open.
>
> The application can now make normal libvirt API calls which are all
> serviced in-process with no RPC layer involved.
>
> It is required to specify an explicit root directory, and locks will be
> acquired on this directory to avoid conflicting with another app that
> might accidentally pick the same directory.
>
> Use of '/' is not explicitly forbidden, but note that the file layout
> used underneath the embedded driver root does not match the file
> layout used by system/session mode drivers. So this cannot be used as
> a backdoor to interact with, or fake, the system/session mode drivers.
>
> Libvirt will create arbitrary files underneath this root directory. The
> root directory can be kept untouched across connection open attempts if
> the application needs persistence. The application is responsible for
> purging everything underneath this root directory when finally no longer
> required.
>
> Even when a virt driver is used in embedded mode, it is still possible
> for it to in turn use functionality that calls out to other secondary
> drivers in libvirtd. For example an embedded instance of QEMU can open
> the network, secret or storage drivers in the system libvirtd.
>
> That said, the application would typically want to at least open an
> embedded secret driver ("secret:///embed?root=/some/path"). Note that
> multiple different embedded drivers can use the same root prefix and
> co-operate just as they would inside a normal libvirtd daemon.
>
> A key thing to note is that for this to work, the application that links
> to libvirt *MUST* be built with -Wl,--export-dynamic to ensure that
> symbols from libvirt.so are exported & thus available to the dynamically
> loaded driver module. If libvirt.so itself was dynamically loaded then
> RTLD_GLOBAL must be passed to dlopen().
>
> Signed-off-by: Daniel P. Berrangé <berrange at redhat.com>
> ---
> src/driver-state.h | 1 +
> src/driver.h | 2 ++
> src/libvirt.c | 72 ++++++++++++++++++++++++++++++++++++++++++++--
> 3 files changed, 73 insertions(+), 2 deletions(-)
>
> diff --git a/src/driver-state.h b/src/driver-state.h
> index 1e2f6ed247..6b3f501e05 100644
> --- a/src/driver-state.h
> +++ b/src/driver-state.h
> @@ -50,6 +50,7 @@ typedef virStateDriver *virStateDriverPtr;
>
> struct _virStateDriver {
> const char *name;
> + bool initialized;
> virDrvStateInitialize stateInitialize;
> virDrvStateCleanup stateCleanup;
> virDrvStateReload stateReload;
> diff --git a/src/driver.h b/src/driver.h
> index ca82ac974b..6278aa05b3 100644
> --- a/src/driver.h
> +++ b/src/driver.h
> @@ -82,6 +82,8 @@ struct _virConnectDriver {
> bool localOnly;
> /* Whether driver needs a server in the URI */
> bool remoteOnly;
> + /* Whether driver can be used in embedded mode */
> + bool embeddable;
> /*
> * NULL terminated list of supported URI schemes.
> * - Single element { NULL } list indicates no supported schemes
> diff --git a/src/libvirt.c b/src/libvirt.c
> index bd2952d036..17b6506faa 100644
> --- a/src/libvirt.c
> +++ b/src/libvirt.c
> @@ -52,6 +52,7 @@
> # include "rpc/virnettlscontext.h"
> #endif
> #include "vircommand.h"
> +#include "virevent.h"
> #include "virfile.h"
> #include "virrandom.h"
> #include "viruri.h"
> @@ -84,6 +85,7 @@
> #ifdef WITH_BHYVE
> # include "bhyve/bhyve_driver.h"
> #endif
> +#include "access/viraccessmanager.h"
>
> #define VIR_FROM_THIS VIR_FROM_NONE
>
> @@ -676,10 +678,12 @@ virStateInitialize(bool privileged,
> return -1;
>
> for (i = 0; i < virStateDriverTabCount; i++) {
> - if (virStateDriverTab[i]->stateInitialize) {
> + if (virStateDriverTab[i]->stateInitialize &&
> + !virStateDriverTab[i]->initialized) {
> virDrvStateInitResult ret;
> VIR_DEBUG("Running global init for %s state driver",
> virStateDriverTab[i]->name);
> + virStateDriverTab[i]->initialized = true;
> ret = virStateDriverTab[i]->stateInitialize(privileged,
> root,
> callback,
> @@ -872,6 +876,7 @@ virConnectOpenInternal(const char *name,
> virConnectPtr ret;
> g_autoptr(virConf) conf = NULL;
> char *uristr = NULL;
> + bool embed = false;
>
> ret = virGetConnect();
> if (ret == NULL)
> @@ -962,6 +967,52 @@ virConnectOpenInternal(const char *name,
> ret->uri) < 0) {
> goto failed;
> }
> +
> + if (STREQ(ret->uri->path, "/embed")) {
> + const char *root = NULL;
> + g_autofree char *regMethod = NULL;
> + VIR_DEBUG("URI path requests %s driver embedded mode",
> + ret->uri->scheme);
> + if (strspn(ret->uri->scheme, "abcdefghijklmnopqrstuvwxyz") !=
> + strlen(ret->uri->scheme)) {
> + virReportError(VIR_ERR_NO_CONNECT,
> + _("URI scheme '%s' for embedded driver is not valid"),
> + ret->uri->scheme);
> + goto failed;
> + }
> +
> + for (i = 0; i < ret->uri->paramsCount; i++) {
> + virURIParamPtr var = &ret->uri->params[i];
> + if (STREQ(var->name, "root"))
> + root = var->value;
> + }
> +
> + if (!root) {
> + virReportError(VIR_ERR_INVALID_ARG, "%s",
> + _("root parameter required for embedded driver"));
> + goto failed;
> + }
> +
> + if (virEventRequireImpl() < 0)
> + goto failed;
> +
> + regMethod = g_strdup_printf("%sRegister", ret->uri->scheme);
> +
> + if (virDriverLoadModule(ret->uri->scheme, regMethod, false) < 0)
> + goto failed;
> +
> + if (virAccessManagerGetDefault() == NULL) {
> + virAccessManagerPtr acl = virAccessManagerNew("none");
> + if (!acl)
> + goto failed;
> + virAccessManagerSetDefault(acl);
> + }
> +
> + if (virStateInitialize(geteuid() == 0, true, root, NULL, NULL) < 0)
> + goto failed;
> +
> + embed = true;
> + }
It would be nice if this logic was moved to a separate function
I've hit a couple issues in testing, not sure if/where the fixes will
live, so I'll just mention them here. Also the reviewed patches are
pushable IMO
I tried this code:
from gi.repository import LibvirtGLib
import libvirt
LibvirtGLib.init(None)
LibvirtGLib.event_register()
conn1 = libvirt.open("qemu:///embed?root=/tmp/foo")
conn2 = libvirt.open("qemu:///embed?root=/tmp/bar")
print(conn1.listAllDomains())
print(conn2.listAllDomains())
With /tmp/foo populated with a VM: both connections see the same values.
So this should be rejected. Even trying to close conn1 fully and open a
new embed root will only see the old root. So maybe this needs knowledge
in the driver lookup.
The second issue: testing with virt-manager, everything locks up with
OpenGraphicsFD:
#0 0x00007ffff7a7f07a in pthread_cond_timedwait@@GLIBC_2.3.2 () at
/lib64/libpthread.so.0
#1 0x00007fffe94be113 in virCondWaitUntil (c=c at entry=0x7fffc8071f98,
m=m at entry=0x7fffc8071ed0, whenms=whenms at entry=1576602286073) at
/home/crobinso/src/libvirt/src/util/virthread.c:159
#2 0x00007fffe44fc549 in qemuDomainObjBeginJobInternal
(driver=driver at entry=0x7fffc8004f30, obj=0x7fffc8071ec0,
job=job at entry=QEMU_JOB_MODIFY,
agentJob=agentJob at entry=QEMU_AGENT_JOB_NONE,
asyncJob=asyncJob at entry=QEMU_ASYNC_JOB_NONE, nowait=nowait at entry=false)
at /home/crobinso/src/libvirt/src/qemu/qemu_domain.c:9357
#3 0x00007fffe4500aa1 in qemuDomainObjBeginJob
(driver=driver at entry=0x7fffc8004f30, obj=<optimized out>,
job=job at entry=QEMU_JOB_MODIFY) at
/home/crobinso/src/libvirt/src/qemu/qemu_domain.c:9521
#4 0x00007fffe4582572 in qemuDomainOpenGraphicsFD (dom=<optimized out>,
idx=<optimized out>, flags=0) at
/home/crobinso/src/libvirt/src/qemu/qemu_driver.c:18990
#5 0x00007fffe968699c in virDomainOpenGraphicsFD (dom=0x7fffd0005830,
idx=0, flags=0) at /home/crobinso/src/libvirt/src/libvirt-domain.c:10664
#6 0x00007fffe98cc6b1 in libvirt_virDomainOpenGraphicsFD () at
/usr/lib64/python3.7/site-packages/libvirtmod.cpython-37m-x86_64-linux-gnu.so
I didn't dig into it any more than that. Otherwise in some mild testing
I'm surprised how much things seemed to 'just work' :)
- Cole
More information about the libvir-list
mailing list