[libvirt] problem with outbound limiting

Stefan Berger stefanb at linux.vnet.ibm.com
Thu Aug 11 13:48:30 UTC 2011


On 08/11/2011 09:39 AM, Stefan Berger wrote:
> On 08/11/2011 08:07 AM, Upendra Moturi wrote:
>> Thanks for your reply.
>>
>> The patch it appears still has problem.
>> I have changed only mtu value as 2kb instead of burst value in the
>> network.c file
>>
>> With average as 2048 , burst as 2048 and peak 2048
>> scp copied at 30-40KB/s
> All test I did were with the patch applied. I saw the throughput 
> collapsing yesterday at 2Mb mtu.
Let me clarify : Yesterday I saw the throughput collapse without the 
patch applied.

    Stefan

>
> I generated a file using
>
> dd if=/dev/urandom of=testfile bs=1024 count=409600
>
> I did the same test you did with 2048 without specifying a specific 
> model of ethernet card and reach 1.9MB/s using scp'ing towards remote 
> machine. When specifying <model type='virtio'/> I also get 1.9MB/s as 
> result.
>> With average as 4096 , burst as 4096 and peak 4096
>> scp copied at same 30-40KB/s
>>
> With these numbers and using the virtio model I ended up getting a 
> throughput of 3.8Mb/s.
>
> The host is a 3.0.0 Linux kernel. Here's the command line output of 
> the tc command inspecting the tap interface of the VM (vnet0).
>
> # tc filter show dev vnet0 root
> filter parent ffff: protocol ip pref 49152 u32
> filter parent ffff: protocol ip pref 49152 u32 fh 800: ht divisor 1
> filter parent ffff: protocol ip pref 49152 u32 fh 800::800 order 2048 
> key ht 800 bkt 0 flowid :1
>   match 00000000/00000000 at 12
>  police 0x5b rate 32768Kbit burst 4Mb mtu 2Kb action drop overhead 0b
> ref 1 bind 1
>
>
> Maybe someone else can verify as well.
>
>    Stefan
>
>> where as earlier it was 1 MB/s.(without patch)
>>
>> On Wed, Aug 10, 2011 at 9:16 PM, Stefan Berger
>> <stefanb at linux.vnet.ibm.com>  wrote:
>>> On 08/10/2011 11:20 AM, Eric Blake wrote:
>>>> On 08/10/2011 09:04 AM, Stefan Berger wrote:
>>>>> On 08/10/2011 02:55 AM, Upendra Moturi wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> does this patch solve your problem? I am setting the MTU to fixed 
>>>>> 2kb.
>>>>> Doing tests with scp seems to indicate that this improved the 
>>>>> situation
>>>>> -- at least for me.
>>>>>
>>>>> Stefan
>>>>>
>>>>> Signed-off-by: Stefan Berger<stefanb at linux.vnet.ibm.com>
>>>>>
>>>>> ---
>>>>> src/util/network.c | 16 ++++++++--------
>>>>> 1 file changed, 8 insertions(+), 8 deletions(-)
>>>>>
>>>>> Index: libvirt-acl/src/util/network.c
>>>>> ===================================================================
>>>>> --- libvirt-acl.orig/src/util/network.c
>>>>> +++ libvirt-acl/src/util/network.c
>>>>> @@ -1156,8 +1156,8 @@ virBandwidthEnable(virBandwidthPtr bandw
>>>>>
>>>>> virCommandFree(cmd);
>>>>> cmd = virCommandNew(TC);
>>>>> - virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent",
>>>>> - "1:", "classid", "1:1", "htb", NULL);
>>>>> + virCommandAddArgList(cmd,"class", "add", "dev", iface, "parent",
>>>>> + "1:", "classid", "1:1", "htb", NULL);
>>>>> virCommandAddArgList(cmd, "rate", average, NULL);
>>>> This hunk was just indentation; you should push the whitespace 
>>>> changes as
>>>> a separate commit (and can push that now, under the trivial rule, 
>>>> without
>>>> needing further review).  [It doesn't help matters that thunderbird 
>>>> botched
>>>> the whitespace in my reply]
>>>>
>>>>> @@ -1202,7 +1202,7 @@ virBandwidthEnable(virBandwidthPtr bandw
>>>>> virCommandAddArgList(cmd, "filter", "add", "dev", iface, "parent",
>>>>> "ffff:", "protocol", "ip", "u32", "match", "ip",
>>>>> "src", "0.0.0.0/0", "police", "rate", average,
>>>>> - "burst", burst, "mtu", burst, "drop", "flowid",
>>>>> + "burst", burst, "mtu", "2kb", "drop", "flowid",
>>>>> ":1", NULL);
>>>> Here's the meat of the proposed patch.  Is 2kb always valid, or 
>>>> will we
>>>> run into problems on NICs that insist on traditional limits near 
>>>> the 1518
>>>> mark?  Should it be user-configurable?  I'm reluctant to ACK this 
>>>> without a
>>>> bit more understanding of why 2kb works, as well as feedback on 
>>>> test results
>>>> with the patch in place.
>>>>
>>> Yes, I am not an exper on tc. I have seen examples with the mtu 
>>> being set to
>>> '1' for dropping icmp traffic for example. So if a VM produces 
>>> smaller than
>>> 2kb packets, which on traditional networks it probably should, then the
>>> packets should go through. We could also leave out the mtu parameter 
>>> above
>>> and it would go to 2kb by default. Check via
>>>
>>> tc filter show dev<ifname>  root
>>>
>>> The 2kb parameter doesn't explain why 2MB doesn't work, though.
>>> Even if this wasn't the final fix, it at least 'improves the 
>>> situation' (for
>>> me).
>>>
>>>     Stefan
>>>
>>>
>>>
>>
>>
>
> -- 
> libvir-list mailing list
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list




More information about the libvir-list mailing list