[Linux-cluster] Piranha not bring up all interfaces

[Lon Hohberger]

On Thu, 2007-01-25 at 13:58 -0500, David Brieck Jr. wrote:
> On 1/25/07, Lon Hohberger <lhh at redhat.com> wrote:
> > On Thu, 2007-01-25 at 09:55 -0500, David Brieck Jr. wrote:
> > > I'm having a problem with our LVS setup using piranha and I'm hoping
> > > this would be the right place to ask about it. What's happening is
> > > that whenever we start or restart pulse, it brings up all the virtual
> > > servers but for some reason it will not bring up all of the interfaces
> > > for the VIP addresses.
> > >
> > > If I just keep restarting pulse it will sometimes bring up different
> > > interfaces and sometimes bring up 6/10 or 8/10 or 9/10 of the
> > > interfaces it is supposed to bring up. However if I manually go back
> > > and issue the ifconfig command to bring up the missing interfaces
> > > everything works just fine.
> > >
> > > There are no errors logged and starting pulse from the command line
> > > with 'pulse -n -v' doesn't indicate anything is wrong either. We have
> > > 10 active virtual servers and each has 3 real servers behind it.
> > >
> > > Any ideas?
> >
> > Happen to be running bonding on e1000?
> >
> > -- Lon
> >
> 
> The system has 4 e1000 ports, 2 are bonded for cluster traffic, one is
> an internet interface and the other is for LAN traffic.
> 
> The interfaces that aren't coming up are on eth3, not bond0.
> 
> I take it this is a known bug?
> 
> The debug output from the command line contains something like this:
> 
> /sbin/ifconfig eth2:1 10.1.1.1 netmask 255.255.255.0 up
> /sbin/ifconfig eth3:1 xx.xx.5.68 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:2 xx.xx.5.69 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:3 xx.xx.5.70 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:4 xx.xx.5.71 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:5 xx.xx.5.77 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:6 xx.xx.5.78 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:7 xx.xx.5.79 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:8 xx.xx.5.81 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:9 xx.xx.5.82 netmask 255.255.255.192 up
> /sbin/ifconfig eth3:11 xx.xx.5.84 netmask 255.255.255.192 up
> /usr/sbin/send_arp

ifconfig ioctl()s (e.g. SIOCGIFCONF, etc.) on 4+ e1000s sometimes fail
for some reason.  I've only seen it with bonding and on RHEL3.  In
clumanager from RHCS3, we worked around it it by allowing users to
switch to RHEL4-ish behavior (e.g. use the iproute2 utilities, which use
the netlink socket instead of ioctls).

The same update was never made for piranha.  Maybe it's needed *shrug*.

I don't know about RHEL4, and I thought it was fixed in RHEL3 U8, but it
might be something to consider...

(Obviously it shouldn't fail).

-- Lon