[libvirt] problem with outbound limiting

[Upendra Moturi]

Thanks for your clarification.
I am using 2.6.38-8-server kernel.
Will test with 3.0.0 kernel


On Thu, Aug 11, 2011 at 7:18 PM, Stefan Berger
<stefanb at linux.vnet.ibm.com> wrote:
> On 08/11/2011 09:39 AM, Stefan Berger wrote:
>>
>> On 08/11/2011 08:07 AM, Upendra Moturi wrote:
>>>
>>> Thanks for your reply.
>>>
>>> The patch it appears still has problem.
>>> I have changed only mtu value as 2kb instead of burst value in the
>>> network.c file
>>>
>>> With average as 2048 , burst as 2048 and peak 2048
>>> scp copied at 30-40KB/s
>>
>> All test I did were with the patch applied. I saw the throughput
>> collapsing yesterday at 2Mb mtu.
>
> Let me clarify : Yesterday I saw the throughput collapse without the patch
> applied.
>
>   Stefan
>
>>
>> I generated a file using
>>
>> dd if=/dev/urandom of=testfile bs=1024 count=409600
>>
>> I did the same test you did with 2048 without specifying a specific model
>> of ethernet card and reach 1.9MB/s using scp'ing towards remote machine.
>> When specifying <model type='virtio'/> I also get 1.9MB/s as result.
>>>
>>> With average as 4096 , burst as 4096 and peak 4096
>>> scp copied at same 30-40KB/s
>>>
>> With these numbers and using the virtio model I ended up getting a
>> throughput of 3.8Mb/s.
>>
>> The host is a 3.0.0 Linux kernel. Here's the command line output of the tc
>> command inspecting the tap interface of the VM (vnet0).
>>
>> # tc filter show dev vnet0 root
>> filter parent ffff: protocol ip pref 49152 u32
>> filter parent ffff: protocol ip pref 49152 u32 fh 800: ht divisor 1
>> filter parent ffff: protocol ip pref 49152 u32 fh 800::800 order 2048 key
>> ht 800 bkt 0 flowid :1
>>  match 00000000/00000000 at 12
>>  police 0x5b rate 32768Kbit burst 4Mb mtu 2Kb action drop overhead 0b
>> ref 1 bind 1
>>
>>
>> Maybe someone else can verify as well.
>>
>>   Stefan
>>
>>> where as earlier it was 1 MB/s.(without patch)
>>>
>>> On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger
>>> <stefanb at linux.vnet.ibm.com>  wrote:
>>>>
>>>> On 08/10/2011 11:20 AM, Eric Blake wrote:
>>>>>
>>>>> On 08/10/2011 09:04 AM, Stefan Berger wrote:
>>>>>>
>>>>>> On 08/10/2011 02:55 AM, Upendra Moturi wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> does this patch solve your problem? I am setting the MTU to fixed 2kb.
>>>>>> Doing tests with scp seems to indicate that this improved the
>>>>>> situation
>>>>>> -- at least for me.
>>>>>>
>>>>>> Stefan
>>>>>>
>>>>>> Signed-off-by: Stefan Berger<stefanb at linux.vnet.ibm.com>
>>>>>>
>>>>>> ---
>>>>>> src/util/network.c | 16 ++++++++--------
>>>>>> 1 file changed, 8 insertions(+), 8 deletions(-)
>>>>>>
>>>>>> Index: libvirt-acl/src/util/network.c
>>>>>> ===================================================================
>>>>>> --- libvirt-acl.orig/src/util/network.c
>>>>>> +++ libvirt-acl/src/util/network.c
>>>>>> @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
>>>>>>
>>>>>> virCommandFree(cmd);
>>>>>> cmd = virCommandNew(TC);
>>>>>> - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent",
>>>>>> - "1:", "classid", "1:1", "htb", NULL);
>>>>>> + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent",
>>>>>> + "1:", "classid", "1:1", "htb", NULL);
>>>>>> virCommandAddArgList(cmd, "rate", average, NULL);
>>>>>
>>>>> This hunk was just indentation; you should push the whitespace changes
>>>>> as
>>>>> a separate commit (and can push that now, under the trivial rule,
>>>>> without
>>>>> needing further review).  [It doesn't help matters that thunderbird
>>>>> botched
>>>>> the whitespace in my reply]
>>>>>
>>>>>> @@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw
>>>>>> virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent",
>>>>>> "ffff:", "protocol", "ip", "u32", "match", "ip",
>>>>>> "src", "0.0.0.0/0", "police", "rate", average,
>>>>>> - "burst", burst, "mtu", burst, "drop", "flowid",
>>>>>> + "burst", burst, "mtu", "2kb", "drop", "flowid",
>>>>>> ":1", NULL);
>>>>>
>>>>> Here's the meat of the proposed patch.  Is 2kb always valid, or will we
>>>>> run into problems on NICs that insist on traditional limits near the
>>>>> 1518
>>>>> mark?  Should it be user-configurable?  I'm reluctant to ACK this
>>>>> without a
>>>>> bit more understanding of why 2kb works, as well as feedback on test
>>>>> results
>>>>> with the patch in place.
>>>>>
>>>> Yes, I am not an exper on tc. I have seen examples with the mtu being
>>>> set to
>>>> '1' for dropping icmp traffic for example. So if a VM produces smaller
>>>> than
>>>> 2kb packets, which on traditional networks it probably should, then the
>>>> packets should go through. We could also leave out the mtu parameter
>>>> above
>>>> and it would go to 2kb by default. Check via
>>>>
>>>> tc filter show dev<ifname>  root
>>>>
>>>> The 2kb parameter doesn't explain why 2MB doesn't work, though.
>>>> Even if this wasn't the final fix, it at least 'improves the situation'
>>>> (for
>>>> me).
>>>>
>>>>    Stefan
>>>>
>>>>
>>>>
>>>
>>>
>>
>> --
>> libvir-list mailing list
>> libvir-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/libvir-list
>
> --
> libvir-list mailing list
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list
>



-- 
Thanks and Regards,
Upendra.M