[libvirt] Unbounded client streams

[Michal Privoznik]

Dear list,

as you might have seen, I've been playing with our streams lately. One 
of the things I wanted to try was how my sparse streams deal with 
vol-download and slow writer (e.g. slow disk). Note to whomever wants to 
try that out: blkio,throttle.write_* is no good for this. So I went the 
old way:

diff --git i/tools/virsh-util.c w/tools/virsh-util.c
index 4b86e29..8f5738d 100644
--- i/tools/virsh-util.c
+++ w/tools/virsh-util.c
@@ -149,6 +149,8 @@ virshStreamSink(virStreamPtr st ATTRIBUTE_UNUSED,
  {
      int *fd = opaque;

+    sleep(60);
+
      return safewrite(*fd, bytes, nbytes);
  }


And found out something ugly: the memory footprint of virsh just keeps 
growing as vol-download progresses. I've tracked down the problem to:

1) The client event loop (virNetClientIOEventLoop) sees some incoming 
data, so it reads it (virNetClientIOHandleInput -> 
virNetClientCallDispatch) and queues the packet 
(virNetClientCallDispatchStream) regardless of the length of already 
queued packets

2) Since the server sees client has read all the data, it sends even 
more data.


And the queue in step 1) just grows and grows. Ideally, the packets are 
taken out from the queue by virStreamRecv (which boils down to 
virNetClientStreamRecvPacket), but that is not on the schedule for the 
next minute.

Frankly, I don't have any idea how to fix this. We can stop reading the 
data (i.e. not set POLLIN on the client <-> server socket) if the queue 
reaches certain length. BUT, this would mean no other call is processed. 
Not even keepalive - and the connection would break.

The other option is to pretend I've never send this e-mail and you've 
read nothing O:-)

Michal