[libvirt] [PATCH] Increased upper limit on lists of pool names

[Jesse Cook]

On Thu, Mar 15, 2012 at 1:10 PM, Eric Blake <eblake at redhat.com> wrote:
> On 03/15/2012 11:48 AM, Daniel P. Berrange wrote:
>> Using the sequence of RPC calls to iterate over this is only
>> addressing that second part of memory usage. So I'm not really
>> feeling that's a huge win, given the complexity it introduces.
>>
>> I'm inclined to say, that if people are creating setups with
>> 1000's of volume & guests, then they can probably spare the
>> extra memory for us to increase the main RPC message payload
>> limit somewhat.
>
> Our own docs/internals/rpc.html.in states:
>
>>       Although the protocol itself defines many arbitrary sized data values in the
>>       payloads, to avoid denial of service attack there are a number of size limit
>>       checks prior to encoding or decoding data. There is a limit on the maximum
>>       size of a single RPC message, limit on the maximum string length, and limits
>>       on any other parameter which uses a variable length array. These limits can
>>       be raised, subject to agreement between client/server, without otherwise
>>       breaking compatibility of the RPC data on the wire.
>
> Increasing limits makes sense, as long as we have a sane way to do it
> while ensuring that on version mismatch, a large packet from the sender
> doesn't crash an older receiver expecting the smaller limit.
>
> In our XDR, should we be converting some of our 'type name<LIST_MAX>'
> over to 'type name<>' notations, to state that they are effectively
> unlimited within the larger bounds of the overall packet size?  Is 256k
> for overall packet size still okay for the problem at hand?
>
> --
> Eric Blake   eblake at redhat.com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

Using the previously posted snippet of code (I'm guessing that the
missing 24 bytes is for RPC header info):

 /* Size of message payload */
 const VIR_NET_MESSAGE_PAYLOAD_MAX = 262120;

We would currently be able to squeeze 7084 things into the RPC message
at 37 bytes per pool.  We could easily reduce this to 33 bytes giving
us 7943 things.  Now I don't know about other users, but this would
give us some room to grow.  Therefore, removing the list limitation
reduces the priority of this from my perspective and gives us the
opportunity to test other limitations that we might run into with
libvirt or otherwise.  When we start moving into the neighborhood of
tens of thousands of things the priority of removing the 256k limit
will be much higher from our perspective, but I'm not sure when that
will be.  I certainly can't speak from the perspective of other users.

-- Jesse