[libvirt] [Qemu-devel] FD passing for chardevs and chardev backend multiplexing

[Eric Blake]

On 12/08/2015 07:59 AM, Daniel P. Berrange wrote:

> So for this my plan is to stop using the QEMU 'file' backend for char
> devs and instead pass across a pre-opened file descriptor, connected
> to virtlogd. There is no "officially documented" way to pass in a
> file descriptor to QEMU chardevs, but since QEMU uses qemu_open(),
> we can make use of the fdset feature to achieve this. eg
> 
> eg, consider fd 33 is the write end of a pipe file descriptor
> I can (in theory) do
> 
>   -add-fd set=2,fd=33 -chardev file,id=charserial0,path=/dev/fdset/2
> 
> Now in practice this doesn't work, because qmp_chardev_open_file()
> passes the O_CREAT|O_TRUNC flags in, which means the qemu_open()
> call will fail when using the pipe FD pased in via fdsets.

Is it just the O_TRUNC that is failing? If so, there is a recent patch
to add an 'append':true flag that switches O_TRUNC off in favor of O_APPEND:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg00762.html

Or is it that the pipe is one-way, but chardev insists on O_RDWR and
fails because it is not two-way?

> 
> After more investigation I found it *is* possible to use a socketpair
> and a pipe backend though...
> 
>   -add-fd set=2,fd=33 -chardev pipe,id=charserial0,path=/dev/fdset/2

Yes, a socketpair is bi-directional, so it supports O_RDWR opening.

> 
> ..because for reasons I don't understand, if QEMU can't open $PATH.in
> and $PATH.out, it'll fallback to just opening $PATH in read-write
> mode even. AFAICT, this is pretty useless with pipes since they
> are unidirectional, but, it works nicely with socketpairs, where
> virtlogd has one of the socketpairs and QEMU gets passed the other
> via fdset.

Is it something where we'd want to support two pipes, and open
/dev/fdset/2 tied to char.in and /dev/fdset/3 tied to char.out, where
uni-directional pipes are again okay?

> 
> I can easily check this works for historical QEMU versions back
> to when fdsets support was added to chardevs, but I'm working if
> the QEMU maintainers consider this usage acceptable over the long
> term, and if so, should we explicitly document it as supported ?

It seems like a bi-directional socketpair as the single endpoint for a
chardev is useful enough to support and document, but I'm not the
maintainer to give final say-so.

> 
> If not, should we introduce a more explicit syntax for passing in
> a pre-opened FD for chardevs ? eg
> 
>   -add-fd set=2,fd=33 -chardev fd,id=charserial0,path=/dev/fdset/2
> 

Difference to the line you tried above:

>   -add-fd set=2,fd=33 -chardev file,id=charserial0,path=/dev/fdset/2

is 'fd' instead of 'file'.  But if we're going to add a new protocol, do
we even need to go through the "/dev/fdset/..." name, or can we just
pass the fd number directly?

> Or just make  -chardev file,id=charserial0,path=/dev/fdset/2 actually
> work ?

I'd lean more to this case - the whole point of fdsets was that we don't
have to add multiple fd protocols; that everyone that understood file
syntax and uses qemu_open() magically gained fd support.

> 
> Or something else ?
> 
> 
> OpenStack has a further requirement to allow use of the serial port
> as an interactive console, at the same time that it is logging to a
> file which is something QEMU can't support at all currently. This
> essentially means being able to have multiple chardev backends all
> connected to the same serial frontend - specifically we would need
> a TCP backend and a file backend concurrently. Again this could be
> implemented in QEMU, but we'd prefer something that works with
> existing QEMU.
> 
> This is not too difficult to achieve with virtlogd really. Instead
> of using the QEMU 'tcp' or 'unix' chardev protocol, we'd just always
> pass QEMU a pre-opened socketpair, and then leave the TCP/UNIX
> socket listening to the virtlogd daemon.
> 
> This is portable with existing QEMU versions, but the obvious downside
> with this is extra copies in the interactive console path. So might it
> be worth exploring the posibility of a chardev multiplexor in QEMU. We
> would still pass in a pre-opened socketpair to QEMU for the logging side
> of things, but would leave the TCP/UNIX socket listening upto QEMU still.
> 
> eg should we make something like this work:
> 
>   -add-fd set=2,fd=33
>   -chardev pipe,id=charserial0file,path=/dev/fdset/2
>   -chardev socket,id=charserial0tcp,host=127.0.0.1,port=9999,telnet,server,nowait
>   -chardev multiplex,id=charserial0,muxA=charserial0file,muxB=charserial1

wouldn't muxB be charserial0tcp (not charserial1)?

>   -serial isa-serial,chardev=charserial0,id=serial0

But the idea of a multiplex protocol that has multiple data sinks (guest
output copied to all sinks) and a single source (at most one source can
provide input to the guest) makes sense on the surface.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 604 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20151208/2a7c3a6b/attachment-0001.sig>