[libvirt] [PATCH v4 06/13] Add support for non-blocking calls in client RPC

[Jiri Denemark]

On Thu, Oct 27, 2011 at 18:05:42 +0200, Jiri Denemark wrote:
> When a client wants to send a keepalive message it needs to do so in a
> non-blocking way to avoid blocking its event loop. This patch adds
> dontBlock flag which says that the call should be processed without
> blocking. Such calls do not have a thread waiting for the result
> associated with them. This means, that sending such call fails if no
> thread is dispatching and writing to the socket would block. In case
> there is a thread waiting for its (normal) call to finish, sending
> non-blocking call just pushes it into the queue and lets the dispatching
> thread send it. The thread which has the buck tries to send all
> non-blocking calls in the queue in a best effort way---if sending them
> would block or there's an error on the socket, non-blocking calls are
> simply removed from the queue and discarded.  In case a non-blocking
> call is partially sent but sending the rest of it would block, it is
> moved into client's unfinishedCall and left for future delivery.  Every
> sending attempt first sends the rest of unfinishedCall and than
> continues with other queued calls.
> ---
> Notes:
>     Version 4:
>     - correctly handle partially sent non-blocking calls that would block
>     
>     Version 3:
>     - no changes
>     
>     Version 2:
>     - no changes
> 
>  src/rpc/virnetclient.c |  261 ++++++++++++++++++++++++++++++++++++++----------
>  1 files changed, 210 insertions(+), 51 deletions(-)
> 
> diff --git a/src/rpc/virnetclient.c b/src/rpc/virnetclient.c
> index 085dc8d..58ba66d 100644
> --- a/src/rpc/virnetclient.c
> +++ b/src/rpc/virnetclient.c
...
> +static void
> +virNetClientDiscardNonBlocking(virNetClientPtr client,
> +                               virNetClientCallPtr thiscall,
> +                               bool error)
> +{
> +    virNetClientCallPtr call = client->waitDispatch;
> +    virNetClientCallPtr prev = NULL;
> +
> +    if (client->unfinishedCall) {
> +        client->unfinishedCall->next = call;
> +        call = client->unfinishedCall;
> +    }
> +
> +    while (call) {
> +        virNetClientCallPtr next = call->next;
> +
> +        if (!call->dontBlock) {
> +            prev = call;
> +            goto skip;
> +        }
> +
> +        /* We can't remove nonblocking call which was already partially sent
> +         * to the remote party (unless there was an error in which case we
> +         * won't be able to send anything anymore anyway); we store it in
> +         * unfinishedCall and when someone needs to send something in the
> +         * future, it will first send the rest of the unfinishedCall.
> +         */
> +        if (!error &&
> +            call->mode != VIR_NET_CLIENT_MODE_COMPLETE &&
> +            call->msg->bufferOffset > 0) {
> +            VIR_DEBUG("Can't finish nonblocking call %p without blocking",
> +                      call);
> +            if (call == client->unfinishedCall)
> +                goto skip;
> +
> +            client->unfinishedCall = call;
> +            goto next;
> +        }

I just realized that this won't work in case virNetClientIOWriteMessage lies
about sent bytes, which it unfortunately does at least for SASL since in that
case it returns zero if [0, bufferLength) bytes were sent and bufferLength
when the last byte of the buffer is sent. It also buffers the data so anytime
virNetClientIOWriteMessage is called, we need to consider the message to be
partially sent even though zero is returned (and bufferOffset is still 0).

Jirka