[libvirt] [PATCH v2] bandwidth: Adjust documentation

John Ferlan jferlan at redhat.com
Tue Feb 18 17:16:52 UTC 2014



On 02/18/2014 05:58 AM, Laine Stump wrote:
> On 02/17/2014 05:45 PM, John Ferlan wrote:
>>
>>
>>>>       <h5><a name="elementVlanTag">Setting VLAN tag (on supported network types only)</a></h5>
>>>> diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in
>>>> index 1ca1bec..d4c390a 100644
>>>> --- a/docs/formatnetwork.html.in
>>>> +++ b/docs/formatnetwork.html.in
>>>> @@ -412,40 +412,81 @@
>>>>
>>>>         <p>
>>>>           The <code><bandwidth></code> element allows setting
>>>> -        quality of service for a particular network.
>>>> -        <span class="since">Since 0.9.4</span> The limits specified
>>>> +        quality of service for a particular network
>>>> +        (<span class="since">since 0.9.4</span>). For a <code>domain</code>
>>>> +        object, the limits specified are applied to the domain traffic.
>>> I'm quite sure about the 'domain traffic'. The <bandwidth/> under domain 
>>> limits the particular <interface/> that has <bandwidth/>. Having 'domain 
>>> traffic' written here may sound like if the domain traffic was 
>>> aggregated and then shaped (which is done in network not in domain).
>>> Maybe 'domain interface traffic'?
>>>
>> Right - I struggled with this one when trying to merge things. Using
>> "domain interface traffic" does target a bit better. Of course the
>> context is discussing network bandwidth, so I guess I just assumed - hah
>> - that the reader would consider the domain traffic as the traffic for
>> that interface.
>>
>> In the end, they both affect the same target interface; however, it
>> wasn't quite clear which would take precedence.  If the interface had
>> one set of values on it, then the domain was defined with a different
>> set, then what happens?
> 
> 
> That is a non-question, because there is no <bandwidth> subelement of a
> <domain>! The <bandwidth> element can appear in 3 places:
> 
> 1) as a subelement of a domain's <interface>. In this case, the
> bandwidth applies to that one interface of that domain.
> 
> 2) as a subelement of a <portgroup> in a <network>. In this case, if a
> domain's <interface> has a "portgroup='xxx'" attribute in its <source>
> element:
> 
>       <interface type='network'>
>         <source network='testnet' portgroup='admin'/>
>         ...
> 
> *and* if the <interface itself has no <bandwidth> element, then the
> <bandwidth> element of the portgroup is applied to that interface.
> 
> 3) as a subelement of a <network>. In this case, the bandwidth specified
> is intended to be to total aggregate bandwidth of all guest interfaces
> attached to that network.
> 
> So, as you can see, there is no concept of "the bandwidth of a domain".

Semantics - I suppose. The "whole" network vs. the "slice" for the
domain. Trying to understand while flipping between two pages and using
"improper" or "inexact" words. I guess what I meant was the bandwidth
assigned to the interface of a domain vs. the bandwidth assigned to the
whole network. In any case, you've answered my questions below...

> 
> The places where there may be a question of precendence:
> 
> 1) if an <interface> has a <bandwidth> *and* it belongs to a portgroup
> that has a <bandwidth>. In this case, we decided that the bandwidth
> under the individual <interface> would take precedence, since at the
> time anyone who had permission to edit the domain (and its <interface>)
> also had permission to edit the network (and its <portgroup>); so
> allowing the specific config for one interface to override the general
> config for many interfaces was more useful.
> 
> 2) if a <network> has a bandwidth defined for the aggregate of all
> interfaces attached to that network, and one interface has bandwidth
> defined that is higher than the aggregate for the entire network. In
> this case, the aggregate bandwidth for the network takes precedence, not
> because of anything we do, but just because the two choke points are
> independent of each other (the <interface> bandwidth control happens on
> the interface's tap device, and the <network> bandwidth control happens
> on the interface part of the bridge device created for that network).
> 
> 
> Did you maybe mean to say
> 
>   If the interface had one set of values on it, then the *NETWORK*
>   was defined with a different set, then what happens?
> 
> 
> ? If so, then I believe (2) above explains in (rather obtusely, of
> course, but I hope it's possible to decipher it)
> 
> 

Your (2) is closer to what I was going for... and yes it makes sense.
The whole portgroup discussion (1) adds complexity to things, but it is
described in the formatnetwork page, although it was in a "different"
paragraph which is why I ignored it at first. I've since moved it (see
below)


>>   Or vice versa - a domain with a set of values
>> applied to an interface and then someone edits the network interface. It
>> seems it would be the "last" set to be applied that would "win".
> 
> What is "interface" vs. "network interface"? I can't tell if you're
> talking about the same thing with two slightly different names, or
> incorrectly naming something in the <network> configuration.

Again some semantics, but it's becoming more clear now - thanks. The
reference to "network interface" was the "<network>" object as opposed
to "an interface"  which was the "<domain><interface>" object (again I
may not be as semantically correct as you expect, but hopefully you see
what I was intending).

> 
> But if you're talking about interface bandwidth vs. network aggregate
> bandwidth, the way that it works is that both sets of rules are
> installed, and the traffic for that guest's interface is shaped twice -
> once as it goes in/out of the guest interface's tap device, then the
> same traffic is shaped again (in aggregate with all other traffic for
> the network) as it goes in/out of the interface that is a part of the
> network's bridge device.
> 

A discussion which I think would be good to add to formatnetwork...

>> Since this is an active topic for other recent patches, hopefully we can
>> come up with a wording that will stick going forward.  I've copied Laine
>> on this response just to bring it to his attention and get his take as
>> well...
>>
> 
> Although I understand the desire to eliminate nearly duplicated wording,
> I'm a bit concerned about trying to combine two things that are almost
> but not entirely the same, due to the potential of the combined
> information being wrong in one of both cases, or at least becoming
> unnecessarily confusing due to extra "exceptions" that are needed to
> explain the differences. In spite of that, I'm fine with it being
> combined - if people complain, we can always split it up again.

Understood; however, considering your last paragraph regarding missing
or not thinking about formatdomain is just cause enough to try and put
the discussion in one place. Keeping things in two places usually
results in one being forgotten during editing. I used cscope to find
relevant bandwidth discussions...


> 
> I can never really form a good opinion about documentation without
> seeing it live on the web page and hearing how others interpret it, and
> my burn-out level on documentation has lately led to my opinion either
> being "yeah, that's okay", or "no, I don't like that, but I don't know
> how to / don't have time to fix it" :-P
> 
> (The recent change to the information for <bandwidth> in the network
> documentation was a side effect of me making a small change to
> validation of network xml to disallow specifying <bandwidth> on network
> types where we can't support setting aggregate bandwidth for the entire
> network. While I was doing that, I noticed that the documentation about
> <bandwidth> in formatnetwork.html.in was obtuse/incorrect, so I made
> some changes to fix those things; I didn't think to look at the
> documentation for <bandwidth> in the domain (<interface>) XML docs,
> because I hadn't changed anything there.)
> 

Because I know it's easier to read without all the git +/-, here's what
I have now in the QoS section on the formatnetwork (hopefully I've
captured everything correctly):


The <bandwidth> element allows setting quality of service for a
particular network (since 0.9.4). Setting bandwidth for a network is
supported only for networks with a <forward> mode of route, nat, or no
mode at all (i.e. an "isolated" network). Setting bandwidth is not
supported for forward modes of bridge, passthrough, private, or hostdev.
Attempts to do this will lead to a failure to define the network or to
create a transient network.

The <bandwidth> element can only be a subelement of a domain's
<interface>, a subelement of a <network>, or a subelement of a
<portgroup> in a <network>.

As a subelement of a domain's <interface>, the bandwidth only applies to
that one interface of the domain. As a subelement of a <network>, the
bandwidth is a total aggregate bandwidth to/from all guest interfaces
attached to that network, not to each guest interface individually. If a
domain's <interface> has <bandwidth> element values higher than the
aggregate for the entire network, then the aggregate bandwidth for the
<network> takes precedence. This is because the two choke points are
independent of each other where the domain's <interface> bandwidth
control is applied on the interface's tap device, while the <network>
bandwidth control is applied on the interface part of the bridge device
created for that network.

As a subelement of a <portgroup> in a <network>, if a domain's
<interface> has a portgroup attribute in its <source> element and if the
<interface> itself has no <bandwidth> element, then the <bandwidth>
element of the portgroup will be applied individually to each guest
interface defined to be a member of that portgroup. Any <bandwidth>
element in the domain's <interface> definition will override the setting
in the portgroup (since 1.0.1).

Incoming and outgoing traffic can be shaped independently. The bandwidth
element can have at most one inbound and at most one outbound child
element. Leaving either of these children elements out results in no QoS
applied for that traffic direction. So, when you want to shape only
incoming traffic, use inbound only, and vice versa. Each of these
elements have one mandatory attribute - average (or floor as described
below). The attributes are as follows, where accepted values for each
attribute is an integer number.

average
    Specifies the desired average bit rate for the interface being
shaped (in kilobytes/second).

peak
    Optional attribute which specifies the maximum rate at which the
bridge can send data (in kilobytes/second). Note the limitation of
implementation: this attribute in the outbound element is ignored (as
Linux ingress filters don't know it yet).

burst
    Optional attribute which specifies the amount of kilobytes that can
be transmitted in a single burst at peak speed.

floor
    Optional attribute available only for the inbound element. This
attribute guarantees minimal throughput for shaped interfaces. This,
however, requires that all traffic goes through one point where QoS
decisions can take place, hence why this attribute works only for
virtual networks for now (that is <interface type='network'/> with a
forward type of route, nat, or no forward at all). Moreover, the virtual
network the interface is connected to is required to have at least
inbound QoS set (average at least). If using the floor attribute users
don't need to specify average. However, peak and burst attributes still
require average. Currently, the Linux kernel doesn't allow ingress
qdiscs to have any classes therefore floor can be applied only on
inbound and not outbound.

Attributes average, peak, and burst are available since 0.9.4, while the
floor attribute is available since 1.0.1.




More information about the libvir-list mailing list