[libvirt] [PATCH] Increased upper limit on lists of pool names

[Eric Blake]

On 03/15/2012 08:41 AM, Daniel P. Berrange wrote:
> On Thu, Mar 15, 2012 at 08:31:32AM -0600, Eric Blake wrote:
>> On 03/14/2012 07:42 PM, Jesse J. Cook wrote:
>>> 256 (8 bits) is insufficient for large scale deployments. 65536 (16 bits) is a
>>> more appropriate limit and should be sufficient. You are more likely to run
>>> into other system limitations first, such as the 31998 inode link limit on
>>> ext3.
>> correctly.  See this comment:
>>
>> /* Maximum length of a memory peek buffer message.
>>  * Note applications need to be aware of this limit and issue multiple
>>  * requests for large amounts of data.
>>  */
>> const REMOTE_DOMAIN_MEMORY_PEEK_BUFFER_MAX = 65536;
>>
>> That is, you cannot send any RPC with more than 64k data, because .
>> With a cap of 256 pool names, that means each pool name can be (on
>> average) about 256 bytes before you hit the RPC cap.
> 
> Not quite right, you meant to look at virnetrpcprotocol.x:
> 
>   /* Size of message payload */
>   const VIR_NET_MESSAGE_PAYLOAD_MAX = 262120;

Okay, I read the wrong constant, so I'm off by a factor of 4.  That
means increasing the pool limit from 256 to 1024 would reduce the
average pool name length from 1024 down to 256, which is still pretty
reasonable; but increasing the pool limit to 64k reduces the average
pool name length limit down to 4 bytes if all names are provided at once
(and those four bytes are probably all consumed in the overhead, since
RPC sends strings by listing length before contents).

For a case study on bumping limits, I only found one commit where we
have done it in the past, using 'git log -p
src/remote/remote_protocol.x' and grepping for '^-.*MAX.*=':

commit 8654175 changed REMOTE_MIGRATE_COOKIE_MAX from 256 to 16k

But there, we were also adding migrate v3, where v2 (and therefore old
servers never sent enough data) to pass the max, so feature negotiation
is built in - a client will only get more than 256 bytes if it requests
migrate v3.

My point remains that bumping limits only delays the limit and also
requires some client feature negotiation, and that the only way to fix
things to unlimited is to break it into multiple RPC calls.

-- 
Eric Blake   eblake at redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 620 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20120315/c1f5063b/attachment-0001.sig>