[libvirt] RFC: automatic setting of ip_forwarding (or not)

[Laine Stump]

  On 10/01/2010 04:44 PM, Zdenek Styblik wrote:
> Uh huh, hi?
>
> On 10/01/2010 08:46 PM, Laine Stump wrote:
> [...]
>> to /etc/sysctl.conf and calling "sysctl -a -p". This gives us the same
>> behavior as currently, but with the advantages that a) our change to the
>> config is documented in /etc/sysctl.conf and b) virtual networked guests
>> won't suddenly have their network fail when some other process runs
>> "sysconfig -a -p".
>>
> You've got me a bit confused here, because what exactly is supposed #
> sysctl -a -p; do? I mean, what kind of magic?

Sorry about that - remove the "-a" from that command.

No magic. As the manpage says, "sysctl -p"  reads the provided file (or 
/etc/sysctl.conf if no file is provided) and sets all the kernel 
parameters listed there to the values given. As you know, 
/etc/sysctl.conf is used, at least on RHEL and Fedora, to "permanently" 
set kernel parameter values to something other than the defaults 
hardcoded into the kernel and loadable kernel modules. (actually they 
can be modified to something else either by individually setting them 
with sysctl -w or by writing directly to the /proc entry, but this will 
be overridden the next time "sysctl -p" is run).


On both RHEL and Fedora, both the network and NetworkManager services 
(and others, eg see the bug reports a bit further down in this reply) 
run sysctl with -p to reload all non-kernel-default settings when they 
are started, and Fedora puts "net.ipv4.ip_forward=0" into 
/etc/sysctl.conf at install time. So at boot time, both of those 
services run "sysctl -p" which sets ip_forward to 0, then libvirtd 
starts, which sets it to 1. At this point, libvirt virtual networks are 
running (as expected), but also there may be some other unwanted routing 
going on (which the admin won't expect). If the network or 
NetworkManager service is later manually restarted (or some other 
parameter is added to sysctl.conf and it's reloaded with "sysctl -p" as 
many HOWTO documents suggest when changing a kernel parameter), 
libvirt's virtual networks suddenly/unexpectedly stop working.


> I've tried this and if I turn on net.ipv4.ip_forward to '1' and run #
> sysctl -a -p; it stays on, unless defined as 0 in /etc/sysctl.conf (and
> sysctl run only with -p, not -a AND -p).

Right, my mistake.


>   And if it's defined as 0 there,
> then there must be reason for being so.


Exactly one of my points. libvirt really wants (no, *needs*) this to be 
on for virtual networks, but it's very likely there was a reason for it 
being turned off, so the admin should at the very least be alerted that 
it's being turned on, or the fact that it's off should be logged in some 
way to assure it gets the admin's attention so they can make the proper 
judgement call.


> Also, why on Earth would you
> have every sysctl parameter in /etc/sysctl.conf unless you're not happy
> with default ones?


Nobody is talking about putting any value there other than 
net.ipv4.ip_forward (which, on RHEL and Fedora is already put there by 
the installer), and that only to assure that it's set to the value that 
libvirtd needs for its virtual networks to work properly. (Possibly my 
erroneous addition of "-a" threw you off there).


> Imho this file is for fine tuning, thus ~ include
> things what I don't like or want changed, but not every possible kernel
> option which I might or might not want to change. Because then there is
> no surprise for such odd behavior.
>
> Also, the outlined problem here sounds more like "left hand doesn't know
> what's doing right hand" or I just didn't get your point.


Again you are correct. currently libvirt is turning it on without 
documenting that fact in the (as far as I know) standard accepted place 
of doing so - /etc/sysctl.conf.


> I'm also
> missing the previous discussion about the problem, if there was any. I
> mean, your e-mail comes out of the blue sky without any previous reference.

Two Active bug reports that I'm aware of offhand (one for RHEL and one 
for Fedora):

   https://bugzilla.redhat.com/show_bug.cgi?id=612865
   https://bugzilla.redhat.com/show_bug.cgi?id=612867

There are also related bugs for older Fedora releases:

   https://bugzilla.redhat.com/show_bug.cgi?id=240922
   https://bugzilla.redhat.com/show_bug.cgi?id=426792
   https://bugzilla.redhat.com/show_bug.cgi?id=467687
   https://bugzilla.redhat.com/show_bug.cgi?id=514228

The first two prompted a short discussion on an internal Red Hat call, 
first a brief mention of it a month or two ago, and again in a similar 
call last week, where the main point of the conversation was "1) this is 
a problem, 2) we should fix it, 3) here's a list of some ideas, 4) we 
need to post these ideas on libvir-list to start a conversation in the 
libvirt community about the best course of action."

I'm not sure how else we could have proceeded that would have made it 
less of a surprise to you. (Of course now that I've hopefully better 
articulated myself, you may be less surprised :-)


> I hope this doesn't sound too much jerk-ish, but I kinda don't like too
> much what I'm reading here.


You may be reading more into this than is there - the discussion is 
newly started, no code has even been written yet (much less patches 
posted, or pushed), and none of the proposed changes are as sweeping as 
you may think.

It sounds like we agree on at least a couple of important points: 1) 
"the left hand should know what the right hand is doing" and 2) if 
ip_forward is set to 0, that was likely done for a reason, so we 
shouldn't silently modify it. it. Beyond that, is a point that is really 
just a fact, not an opinion: 3) libvirt's virtual networks won't work 
unless ip_forward is turned on. The current code is only compatible with 
1 of those 3; we need to agree on a method to satisfy all of them. Are 
you voting for my proposal (1) in the original mail? (do nothing), or do 
you have another idea?