[libvirt-users] Need to increase the rx and tx buffer size of my interface
Ashish Kurian
ashishbnv at gmail.com
Thu Oct 26 07:38:49 UTC 2017
Hi Yalan,
In the previous email you mentioned "tx_queue_size='512' will not work in
the guest with direct type interface, in fact, no matter what you set, it
will not work and guest will get the default '256'. "
So if I am using macvtap for my interfaces, then the device type will
always be direct type. Does it mean that there is no way I can increase the
buffer size with the macvtap interfaces?
Best Regards,
Ashish Kurian
On Thu, Oct 26, 2017 at 9:04 AM, Ashish Kurian <ashishbnv at gmail.com> wrote:
> Hi Yalan,
>
> Thank you for your comment on qemu-kvm-rhev
>
> I am waiting for a response about my previous email with the logs
> attached. I do not understand what is the problem.
>
>
> On Oct 26, 2017 8:58 AM, "Yalan Zhang" <yalzhang at redhat.com> wrote:
>
> Hi Ashish,
>
> Please never mind for qemu-kvm-rhev.
> qemu with the code base 2.10.0 will support the tx_queue_size and
> rx_queue_size.
>
> Thank you~
>
>
>
>
>
> -------
> Best Regards,
> Yalan Zhang
> IRC: yalzhang
> Internal phone: 8389413
>
> On Thu, Oct 26, 2017 at 2:22 PM, Yalan Zhang <yalzhang at redhat.com> wrote:
>
>> Hi Ashish,
>>
>> Are these packages available for free? How can I install them?
>> => You did have vhost backend driver. Do not set <driver name='qemu'...>,
>> by default it will use vhost as backend driver.
>>
>> Is it possible to have my interfaces with an IP address inside the VM to
>> be bridged to the physical interfaces on the host?
>> => Yes, you can create a linux bridge with physical interface connected,
>> and use bridge type interface. Refer to https://libvirt.org/formatd
>> omain.html#elementsNICSBridge
>> direct type is also ok (but your host and guest have no access to each
>> other).
>>
>> Is it also a possibility that I change the rx and tx buffer on the
>> physical interface on the host and it is reflected automatically inside the
>> VM as you said it will always receive the default value of the host?
>> => No, it do not receive the default value of the host. It's the default
>> value related with the virtual device driver on the guest.
>> hostdev type interface will passthrough the physical interface or VF of
>> the host to guest, it will get the device's parameters for rx and tx buffer.
>>
>>
>>
>> -------
>> Best Regards,
>> Yalan Zhang
>> IRC: yalzhang
>> Internal phone: 8389413
>>
>> On Thu, Oct 26, 2017 at 1:30 PM, Ashish Kurian <ashishbnv at gmail.com>
>> wrote:
>>
>>> Hi Yalan,
>>>
>>> Thank you for your response. I do not have the following packages
>>> installed
>>>
>>> vhost backend driver
>>> qemu-kvm-rhev package
>>>
>>> Are these packages available for free? How can I install them?
>>>
>>> In my KVM VM, I must have an IP address to the interfaces that I am
>>> trying to increasing the buffers. That is the reason I was using macvtap
>>> (direct type interface). Is it possible to have my interfaces with an IP
>>> address inside the VM to be bridged to the physical interfaces on the host?
>>>
>>> Is it also a possibility that I change the rx and tx buffer on the
>>> physical interface on the host and it is reflected automatically inside the
>>> VM as you said it will always receive the default value of the host?
>>>
>>>
>>> Best Regards,
>>> Ashish Kurian
>>>
>>> On Thu, Oct 26, 2017 at 6:45 AM, Yalan Zhang <yalzhang at redhat.com>
>>> wrote:
>>>
>>>> Hi Ashish,
>>>>
>>>> I have tested with your xml in the first mail, and it works for rx_queue_size(see
>>>> below).
>>>> multiqueue need to work with vhost backend driver. And when you set
>>>> "queues=1" it will ignored.
>>>>
>>>> Please check your qemu-kvm-rhev package, should be newer than
>>>> qemu-kvm-rhev-2.9.0-16.el7_4.2
>>>> And the logs?
>>>>
>>>> tx_queue_size='512' will not work in the guest with direct type
>>>> interface, in fact, no matter what you set, it will not work and guest will
>>>> get the default '256'.
>>>> We only support vhost-user backend to have more than 256. refer to
>>>> https://libvirt.org/formatdomain.html#elementsNICSEthernet
>>>>
>>>> tx_queue_size
>>>> The optional tx_queue_size attribute controls the size of virtio ring
>>>> for each queue as described above. The default value is hypervisor
>>>> dependent and may change across its releases. Moreover, some hypervisors
>>>> may pose some restrictions on actual value. For instance, QEMU v2.9
>>>> requires value to be a power of two from [256, 1024] range. In addition to
>>>> that, this may work only for a subset of interface types, e.g.
>>>> aforementioned QEMU enables this option only for vhostuser type. Since
>>>> 3.7.0 (QEMU and KVM only)
>>>> multiqueue only supports vhost as backend driver.
>>>>
>>>> # rpm -q libvirt qemu-kvm-rhev
>>>> libvirt-3.2.0-14.el7_4.3.x86_64
>>>> qemu-kvm-rhev-2.9.0-16.el7_4.9.x86_64
>>>>
>>>> 1. the xml as below
>>>> <interface type='direct'>
>>>> <mac address='52:54:00:00:b5:99'/>
>>>> <source dev='eno1' mode='vepa'/>
>>>> <model type='virtio'/>
>>>> <driver name='vhost' queues='5' rx_queue_size='512'
>>>> tx_queue_size='512'>
>>>> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off'
>>>> ufo='off' mrg_rxbuf='off'/>
>>>> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
>>>> </driver>
>>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
>>>> function='0x0'/>
>>>> </interface>
>>>>
>>>> 2. after start the vm, check the qemu command line:
>>>> *-netdev
>>>> tap,fds=26:28:29:30:31,id=hostnet0,vhost=on,vhostfds=32:33:34:35:36*
>>>> -device virtio-net-pci,csum=off,gso=off,host_tso4=off,host_tso6=off,
>>>> host_ecn=off,host_ufo=off,mrg_rxbuf=off,guest_csum=off,guest
>>>> _tso4=off,guest_tso6=off,guest_ecn=off,guest_ufo=off,
>>>> *mq=on,vectors=12,rx_queue_size=512,tx_queue_size=512*,netdev=hostnet
>>>> 0,id=net0,mac=52:54:00:00:b5:99,bus=pci.0,addr=0x3
>>>>
>>>> 3. check on guest
>>>> # ethtool -g eth0
>>>> Ring parameters for eth0:
>>>> Pre-set maximums:
>>>> RX: *512 ==> rx_queue_size works*
>>>> RX Mini: 0
>>>> RX Jumbo: 0
>>>> TX: *256 ===> no change*
>>>> Current hardware settings:
>>>> RX: *512 **==> rx_queue_size works*
>>>> RX Mini: 0
>>>> RX Jumbo: 0
>>>> TX: *256 ===> no change*
>>>>
>>>> # ethtool -l eth0
>>>> Channel parameters for eth0:
>>>> Pre-set maximums:
>>>> RX: 0
>>>> TX: 0
>>>> Other: 0
>>>> Combined: *5 ==> queues what we set*
>>>> Current hardware settings:
>>>> RX: 0
>>>> TX: 0
>>>> Other: 0
>>>> Combined: 1
>>>>
>>>>
>>>> If change to qemu as driver,
>>>> # virsh edit rhel7
>>>> ..
>>>> <interface type='direct'>
>>>> <mac address='52:54:00:00:b5:99'/>
>>>> <source dev='eno1' mode='vepa'/>
>>>> <model type='virtio'/>
>>>> <driver name='qemu' queues='5' rx_queue_size='512'
>>>> tx_queue_size='512'>
>>>> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off'
>>>> ufo='off' mrg_rxbuf='off'/>
>>>> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
>>>> </driver>
>>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
>>>> function='0x0'/>
>>>> </interface>
>>>> ..
>>>> Domain rhel7 XML configuration edited. ==> the xml can validate and save
>>>>
>>>> # virsh start rhel7
>>>> Domain rhel7 started
>>>>
>>>>
>>>> # virsh dumpxml rhel7 | grep /interface -B9
>>>> <source dev='eno1' mode='vepa'/>
>>>> <target dev='macvtap0'/>
>>>> <model type='virtio'/>
>>>> *<driver name='qemu' queues='5' rx_queue_size='512'
>>>> tx_queue_size='512'>*
>>>> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off'
>>>> ufo='off' mrg_rxbuf='off'/>
>>>> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
>>>> </driver>
>>>> <alias name='net0'/>
>>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
>>>> function='0x0'/>
>>>> </interface>
>>>>
>>>>
>>>> * -netdev tap,fds=26:28:29:30:31*,id=hostnet0 -device
>>>> virtio-net-pci,csum=off,gso=off,host_tso4=off,host_tso6=off,
>>>> host_ecn=off,host_ufo=off,mrg_rxbuf=off,guest_csum=off,guest
>>>> _tso4=off,guest_tso6=off,guest_ecn=off,guest_ufo=off,
>>>> *rx_queue_size=512,tx_queue_size=512*,netdev=hostnet0,id=net0,mac=52:
>>>> 54:00:00:b5:99,bus=pci.0,addr=0x3
>>>>
>>>> *"mq=on,vectors=12" is missing*, indicates there is no multiqueue
>>>>
>>>> and check on guest
>>>>
>>>> # ethtool -l eth0
>>>> Channel parameters for eth0:
>>>> Pre-set maximums:
>>>> RX: 0
>>>> TX: 0
>>>> Other: 0
>>>> Combined: 1 ==> no multiqueue
>>>> Current hardware settings:
>>>> RX: 0
>>>> TX: 0
>>>> Other: 0
>>>> Combined: 1
>>>>
>>>> # ethtool -g eth0
>>>> Ring parameters for eth0:
>>>> Pre-set maximums:
>>>> RX: *512*
>>>> RX Mini: 0
>>>> RX Jumbo: 0
>>>> TX: 256
>>>> Current hardware settings:
>>>> RX: *512*
>>>> RX Mini: 0
>>>> RX Jumbo: 0
>>>> TX: 256
>>>>
>>>>
>>>>
>>>>
>>>> -------
>>>> Best Regards,
>>>> Yalan Zhang
>>>> IRC: yalzhang
>>>> Internal phone: 8389413
>>>>
>>>> On Thu, Oct 26, 2017 at 2:33 AM, Ashish Kurian <ashishbnv at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Michal,
>>>>>
>>>>> An update to what I have already said : when I try adding <driver
>>>>> name='qemu' txmode='iothread' ioeventfd='on' event_idx='off' queues='1'
>>>>> rx_queue_size='512' tx_queue_size='512'> although it showed me the error as
>>>>> mentioned, when I checked the xml again I saw that <driver
>>>>> name='qemu' txmode='iothread' ioeventfd='on' event_idx='off' > is added to
>>>>> the interface.
>>>>>
>>>>> The missing parameters are : queues='1' rx_queue_size='512'
>>>>> tx_queue_size='512'
>>>>>
>>>>> Best Regards,
>>>>> Ashish Kurian
>>>>>
>>>>> On Wed, Oct 25, 2017 at 5:07 PM, Ashish Kurian <ashishbnv at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Michal,
>>>>>>
>>>>>> What I found was that when I restarted the machine and did a virsh
>>>>>> edit command to see the xml config, I see that it is was not actually
>>>>>> changed. This suggests why I saw 256 again after restarting.
>>>>>>
>>>>>> So now I tried again to edit the xml via virsh edit command and used
>>>>>> the following to set the parameters.
>>>>>>
>>>>>> <driver name='qemu' txmode='iothread' ioeventfd='on' event_idx='off'
>>>>>> queues='1' rx_queue_size='512' tx_queue_size='512'>
>>>>>> </driver>
>>>>>>
>>>>>> It was not accepted and I got the error saying :
>>>>>>
>>>>>>
>>>>>> error: XML document failed to validate against schema: Unable to
>>>>>> validate doc against /usr/share/libvirt/schemas/domain.rng
>>>>>> Extra element devices in interleave
>>>>>> Element domain failed to validate content
>>>>>>
>>>>>> What does this imply? I have two more other interfaces and do I have
>>>>>> to the same to them also?
>>>>>>
>>>>>> Btw, there are now logs generated now in the domain log or libvirtd
>>>>>> log
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Best Regards,
>>>>>> Ashish Kurian
>>>>>>
>>>>>> On Wed, Oct 25, 2017 at 2:50 PM, Michal Privoznik <
>>>>>> mprivozn at redhat.com> wrote:
>>>>>>
>>>>>>> On 10/25/2017 01:53 PM, Ashish Kurian wrote:
>>>>>>> > Dear Users/Developers,
>>>>>>> >
>>>>>>> > I am using a KVM Ubuntu VM as a degrader to apply specific delays
>>>>>>> to
>>>>>>> > incoming packets. As the delay for my packets can be higher than
>>>>>>> 7.5
>>>>>>> > seconds, there is not enough buffer on my interface to buffer all
>>>>>>> the
>>>>>>> > packets. Therefore those overflowing packets are dropped in the
>>>>>>> machine and
>>>>>>> > not forwarded.
>>>>>>> >
>>>>>>> > When I tried to use the command ethtool -G ens8 rx 512 to
>>>>>>> increase the
>>>>>>> > buffer size, I get the following error.
>>>>>>> >
>>>>>>> > Cannot set device ring parameters: Operation not permitted
>>>>>>> >
>>>>>>> > I have kept the VM xml files as specified in the link :
>>>>>>> > https://libvirt.org/formatdomain.html. The value that I kept in
>>>>>>> my xml file
>>>>>>> > is as follows.
>>>>>>> >
>>>>>>> > <interface type='direct'>
>>>>>>> > <mac address='52:54:00:72:f9:eb'/>
>>>>>>> > <source dev='enp7s0f0' mode='vepa'/>
>>>>>>> > <model type='virtio'/>
>>>>>>> > <driver name='vhost' queues='5' rx_queue_size='512'
>>>>>>> > tx_queue_size='512'>
>>>>>>> > <host csum='off' gso='off' tso4='off' tso6='off' ecn='off'
>>>>>>> ufo='off'
>>>>>>> > mrg_rxbuf='off'/>
>>>>>>> > <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
>>>>>>> > </driver>
>>>>>>> > <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>>>>>>> > function='0x0'/>
>>>>>>> > </interface>
>>>>>>> > <interface type='direct'>
>>>>>>> > <mac address='52:54:00:00:b5:99'/>
>>>>>>> > <source dev='enp7s0f1' mode='vepa'/>
>>>>>>> > <model type='virtio'/>
>>>>>>> > <driver name='vhost' queues='5' rx_queue_size='512'
>>>>>>> > tx_queue_size='512'>
>>>>>>> > <host csum='off' gso='off' tso4='off' tso6='off' ecn='off'
>>>>>>> ufo='off'
>>>>>>> > mrg_rxbuf='off'/>
>>>>>>> > <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/>
>>>>>>> > </driver>
>>>>>>> > <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
>>>>>>> > function='0x0'/>
>>>>>>>
>>>>>>> So what does the qemu command line look like? You can find it in
>>>>>>> either
>>>>>>> libvirtd log or domain log.
>>>>>>>
>>>>>>> http://wiki.libvirt.org/page/DebugLogs
>>>>>>>
>>>>>>> Michal
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> libvirt-users mailing list
>>>>> libvirt-users at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/libvirt-users
>>>>>
>>>>
>>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20171026/0cacd99c/attachment.htm>
More information about the libvirt-users
mailing list