[libvirt] problem with outbound limiting

[Stefan Berger]

On 08/11/2011 09:54 AM, Upendra Moturi wrote:
> Thanks for your clarification.
> I am using 2.6.38-8-server kernel.
> Will test with 3.0.0 kernel
Also works on 2.6.38.8-35.fc15.x86_64 kernel.

    Stefan

>
> On Thu, Aug 11, 2011 at 7:18 PM, Stefan Berger
> <stefanb at linux.vnet.ibm.com>  wrote:
>> On 08/11/2011 09:39 AM, Stefan Berger wrote:
>>> On 08/11/2011 08:07 AM, Upendra Moturi wrote:
>>>> Thanks for your reply.
>>>>
>>>> The patch it appears still has problem.
>>>> I have changed only mtu value as 2kb instead of burst value in the
>>>> network.c file
>>>>
>>>> With average as 2048 , burst as 2048 and peak 2048
>>>> scp copied at 30-40KB/s
>>> All test I did were with the patch applied. I saw the throughput
>>> collapsing yesterday at 2Mb mtu.
>> Let me clarify : Yesterday I saw the throughput collapse without the patch
>> applied.
>>
>>    Stefan
>>
>>> I generated a file using
>>>
>>> dd if=/dev/urandom of=testfile bs=1024 count=409600
>>>
>>> I did the same test you did with 2048 without specifying a specific model
>>> of ethernet card and reach 1.9MB/s using scp'ing towards remote machine.
>>> When specifying<model type='virtio'/>  I also get 1.9MB/s as result.
>>>> With average as 4096 , burst as 4096 and peak 4096
>>>> scp copied at same 30-40KB/s
>>>>
>>> With these numbers and using the virtio model I ended up getting a
>>> throughput of 3.8Mb/s.
>>>
>>> The host is a 3.0.0 Linux kernel. Here's the command line output of the tc
>>> command inspecting the tap interface of the VM (vnet0).
>>>
>>> # tc filter show dev vnet0 root
>>> filter parent ffff: protocol ip pref 49152 u32
>>> filter parent ffff: protocol ip pref 49152 u32 fh 800: ht divisor 1
>>> filter parent ffff: protocol ip pref 49152 u32 fh 800::800 order 2048 key
>>> ht 800 bkt 0 flowid :1
>>>   match 00000000/00000000 at 12
>>>   police 0x5b rate 32768Kbit burst 4Mb mtu 2Kb action drop overhead 0b
>>> ref 1 bind 1
>>>
>>>
>>> Maybe someone else can verify as well.
>>>
>>>    Stefan
>>>
>>>> where as earlier it was 1 MB/s.(without patch)
>>>>
>>>> On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger
>>>> <stefanb at linux.vnet.ibm.com>    wrote:
>>>>> On 08/10/2011 11:20 AM, Eric Blake wrote:
>>>>>> On 08/10/2011 09:04 AM, Stefan Berger wrote:
>>>>>>> On 08/10/2011 02:55 AM, Upendra Moturi wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> does this patch solve your problem? I am setting the MTU to fixed 2kb.
>>>>>>> Doing tests with scp seems to indicate that this improved the
>>>>>>> situation
>>>>>>> -- at least for me.
>>>>>>>
>>>>>>> Stefan
>>>>>>>
>>>>>>> Signed-off-by: Stefan Berger<stefanb at linux.vnet.ibm.com>
>>>>>>>
>>>>>>> ---
>>>>>>> src/util/network.c | 16 ++++++++--------
>>>>>>> 1 file changed, 8 insertions(+), 8 deletions(-)
>>>>>>>
>>>>>>> Index: libvirt-acl/src/util/network.c
>>>>>>> ===================================================================
>>>>>>> --- libvirt-acl.orig/src/util/network.c
>>>>>>> +++ libvirt-acl/src/util/network.c
>>>>>>> @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
>>>>>>>
>>>>>>> virCommandFree(cmd);
>>>>>>> cmd = virCommandNew(TC);
>>>>>>> - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent",
>>>>>>> - "1:", "classid", "1:1", "htb", NULL);
>>>>>>> + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent",
>>>>>>> + "1:", "classid", "1:1", "htb", NULL);
>>>>>>> virCommandAddArgList(cmd, "rate", average, NULL);
>>>>>> This hunk was just indentation; you should push the whitespace changes
>>>>>> as
>>>>>> a separate commit (and can push that now, under the trivial rule,
>>>>>> without
>>>>>> needing further review).  [It doesn't help matters that thunderbird
>>>>>> botched
>>>>>> the whitespace in my reply]
>>>>>>
>>>>>>> @@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw
>>>>>>> virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent",
>>>>>>> "ffff:", "protocol", "ip", "u32", "match", "ip",
>>>>>>> "src", "0.0.0.0/0", "police", "rate", average,
>>>>>>> - "burst", burst, "mtu", burst, "drop", "flowid",
>>>>>>> + "burst", burst, "mtu", "2kb", "drop", "flowid",
>>>>>>> ":1", NULL);
>>>>>> Here's the meat of the proposed patch.  Is 2kb always valid, or will we
>>>>>> run into problems on NICs that insist on traditional limits near the
>>>>>> 1518
>>>>>> mark?  Should it be user-configurable?  I'm reluctant to ACK this
>>>>>> without a
>>>>>> bit more understanding of why 2kb works, as well as feedback on test
>>>>>> results
>>>>>> with the patch in place.
>>>>>>
>>>>> Yes, I am not an exper on tc. I have seen examples with the mtu being
>>>>> set to
>>>>> '1' for dropping icmp traffic for example. So if a VM produces smaller
>>>>> than
>>>>> 2kb packets, which on traditional networks it probably should, then the
>>>>> packets should go through. We could also leave out the mtu parameter
>>>>> above
>>>>> and it would go to 2kb by default. Check via
>>>>>
>>>>> tc filter show dev<ifname>    root
>>>>>
>>>>> The 2kb parameter doesn't explain why 2MB doesn't work, though.
>>>>> Even if this wasn't the final fix, it at least 'improves the situation'
>>>>> (for
>>>>> me).
>>>>>
>>>>>     Stefan
>>>>>
>>>>>
>>>>>
>>>>
>>> --
>>> libvir-list mailing list
>>> libvir-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/libvir-list
>> --
>> libvir-list mailing list
>> libvir-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/libvir-list
>>
>
>