[libvirt] Raising limits for our RPC messages

[Michal Privoznik]

Dear list,

maybe you've seen us raising limit for various parts of RPC messages
(for eaxmple: d15b29be, 66bfc7cc61ca0, e914dcfd, etc.). It usually
happens when we receive a report that current limits are not enough.
Well, we just did:

https://bugzilla.redhat.com/show_bug.cgi?id=1440683

The thing is, virConnectGetAllDomainStats() is unable to encode RPC
message because there's so much domains & stats to send that the limit
is reached. Now, I was thinking how to approach this problem. Yes, we
can raise the limits again. But we also have another approach: split
data into multiple messages. The current limit for an RPC message is
4MB. Say you have 9MB of data you want to send to the other side. We
could construct 3 messages (4 + 4 + 1) and send them to the other side
where the data would be reconstructed.
At RPC level, we could use @status header file to represent intermediate
messages. That is, for the example above messages with the following
headers would be constructed:

{.len = 4MB, prog, vers, proc, .type = VIR_NET_REPLY, .serial, .status =
VIR_NET_CONTINUE}
{.len = 4MB, prog, vers, proc, .type = VIR_NET_REPLY, .serial, .status =
VIR_NET_CONTINUE}
{.len = 1MB, prog, vers, proc, .type = VIR_NET_REPLY, .serial, .status =
VIR_NET_OK}

(the actual @len accounts for message header too, so the message would
be split into <nearly 4MB + nearly 4MB + slightly over 1MB> chunks, but
that is not important right now)

Just like we have in streams, VIR_NET_CONTINUE tells that there is more
data yet to come, VIR_NET_OK could then mark the last packet in the stream.

Cool you say, this could work. But not really. The problem with this
approach is:

a) we just transform problem to a different one (the limit of number of
messages data can be split into)

b) increased overhead (message header is sent multiple times)

c) I don't expect we will ever support cherry-picking of messages
(caller picks just the message that contains data they are interested in
and discards the rest).

So after all, we will effectively have the limit of message size
increased to $limit_of_split_messages * $current_limit_of_message_size.
Instead of introducing complex code that splits data into messages and
reconstructs back, we might as well increase the current limit of
message size.

Michal