nanny "Read Timed Out" Errors
Tapan Thapa
tapan.thapa2000 at gmail.com
Fri Nov 6 03:55:34 UTC 2009
Hello Mike,
Now your network and config file looks perfectly ok.
As you are using direct routing so your real server (kiwidev4) has two
options.
1st - Use iptables to redirect traffic destined by linux director to real
server.
2nd - Use arptables_jf method. (yum install arptables_jf).
As you told that you can not modify iptables rules as kiwidev4 machine is
being used for other purpose as well so you have to use method 2
(arptables_jf).
Below are the steps which you need to performed on real server/servers only:
1. Install arptables_jf rpm from redhat distribution cd or from internet
into kiwidev4.
2. arptables -A IN -d 192.168.3.40 -j DROP
3. arptables -A OUT -d 192.168.3.40 -j mangle --mangle-ip-s 192.168.3.38
4. service arptables_jf save
5. chkconfig arptables_jf on
6. ifconfig eth0:1 192.168.3.40 netmask 255.255.248.0 broadcast
192.168.3.255 up
(For subsequent reboot, you will have to create a file under
/etc/sysconfig/network-scripts/ifcfg-eth0:1) and need to put values for vip
(192.168.3.40) address.
Then restart httpd service in kiwidev4 and let me know if it works.
Please also post ipvsadm -L -n output after this... :)
Regards
Tapan
India
On Fri, Nov 6, 2009 at 12:23 AM, mojorising <moj0rising at aim.com> wrote:
> Okay. It seems I missed a critical piece of my config file when I
> copy/pasted it to you. Sorry about that.
>
> So here is my ha.cf file now
>
> [root at omsbuild ~]# cat /etc/sysconfig/ha/lvs.cf
> serial_no = 93
> primary = 192.168.3.28
> service = lvs
> backup = 0.0.0.0
> heartbeat = 1
> heartbeat_port = 539
> keepalive = 6
> deadtime = 18
> network = direct
> debug_level = NONE
> virtual test1 {
> active = 1
> address = 192.168.3.40 eth0:1
> vip_nmask = 255.255.248.0
> port = 80
> expect = "HTTP"
> use_regex = 0
> load_monitor = none
> scheduler = lc
> protocol = tcp
> timeout = 6
> reentry = 15
> quiesce_server = 0
> server kiwidev4 {
> address = 192.168.3.38
> active = 1
> port = 80
> weight = 1
> }
> }
>
>
> I took out those other machines because I can not change their IPs
> (I'm just using them for testing). So in their place, I put a machine
> (kiwidev4) that happens to be on the same subnet as the LVS box.
> kiwidev4 was always there and active but that part of the config file
> was accidentally clipped off from my message. :(
>
> I can not change those iptables rules at this time because that
> kiwidev4 box may be in use for some other testing at the moment. Can
> we do this without making the specified changes to iptables? It seems
> we shouldn't need to do that. I will eventually be using LVS to
> balance traffic to Windows machines as well so I need to be able to do
> without iptables for that reason also.
>
>
> Mike
>
>
> On 04/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
> > Hello Mike,
> >
> > Now your network status looks good.
> >
> > But still i can not see any real server on the same network. i.e.
> > 192.168.3.x.
> >
> > As per your lvs.cf, you have configured two real servers. First one is
> > server Speedy and second one is server test1 and currently both are not
> > active. (active = 0) They should be (active = 1).
> >
> > And also your real servers are not in right subnet.
> >
> > Your real server should be on same 192.168.3.x network.
> >
> > Your example lvs.cf should look like:
> >
> > serial_no = 93
> > primary = 192.168.3.28
> > service = lvs
> > backup = 0.0.0.0
> > heartbeat = 1
> > heartbeat_port = 539
> > keepalive = 6
> > deadtime = 18
> > network = direct
> > debug_level = NONE
> > virtual test1 {
> > active = 1
> > address = 192.168.3.40 eth0:1
> > vip_nmask = 255.255.248.0
> > port = 80
> > expect = "HTTP"
> > use_regex = 0
> > load_monitor = none
> > scheduler = lc
> > protocol = tcp
> > timeout = 6
> > reentry = 15
> > quiesce_server = 0
> > server Speedy {
> > address = 192.168.3.29
> > active = 1
> > port = 80
> > weight = 1
> > }
> > server test1 {
> > address = 192.168.3.30
> > active = 1
> > port = 80
> > weight = 1
> > }
> >
> > Please change the ip address of Speedy server to 192.168.3.29 and test1
> > server to 192.168.3.30 with subnet mask of 255.255.248.0 and restart
> > network
> > and httpd service.
> >
> > then fire below mentioned commands in both real servers (Not in lvs
> > server):
> >
> > chkconfig iptables on
> > iptables -F
> > iptables -t nat -A PREROUTING -p tcp --dport 80 -d 192.168.3.40 -j
> REDIRECT
> > service iptables save
> >
> > then please restart pulse service at linux director server (lvs server)
> and
> > wait for 2 minutes. and then check the status of ipvsadm -L -n command
> and
> > let me know in case any issues.
> >
> >
> > Regards
> > Tapan Thapa
> > India
> >
> >
> >
> > On Thu, Nov 5, 2009 at 1:25 AM, mojorising <moj0rising at aim.com> wrote:
> >
> >> Tapan, sorry for confusing you. I overlooked my virtual IP and
> >> accidedntally left it with an IP on the wrong net. This is now
> >> corrected.
> >>
> >> Those real servers on other nets are still in my configuration but
> >> they are "down," as they were before. I do have one real server up on
> >> the proper net -- 192.168.3.38.
> >>
> >>
> >> My present network interface set-up:
> >>
> >> eth0 Link encap:Ethernet HWaddr 00:50:56:AE:14:E3
> >> inet addr:192.168.3.28 Bcast:192.168.7.255 Mask:255.255.248.0
> >> inet6 addr: fe80::250:56ff:feae:14e3/64 Scope:Link
> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> RX packets:36180876 errors:1122 dropped:1234 overruns:0
> frame:0
> >> TX packets:8729361 errors:0 dropped:0 overruns:0 carrier:0
> >> collisions:0 txqueuelen:1000
> >> RX bytes:72196093 (68.8 MiB) TX bytes:610192805 (581.9 MiB)
> >> Interrupt:177 Base address:0x1400
> >>
> >> eth0:1 Link encap:Ethernet HWaddr 00:50:56:AE:14:E3
> >> inet addr:192.168.3.40 Bcast:192.168.7.255
> Mask:255.255.248.0
> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> Interrupt:177 Base address:0x1400
> >>
> >>
> >> A problem is that I am still getting those error messages from nanny:
> >>
> >> Nov 4 11:06:52 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed
> out
> >> Nov 4 11:06:53 omsbuild nanny[20548]: READ to 192.168.3.38:80 timed
> out
> >> Nov 4 11:07:04 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed
> out
> >> Nov 4 11:07:05 omsbuild nanny[20548]: READ to 192.168.3.38:80 timed
> out
> >>
> >>
> >> My lvs.cf file now:
> >>
> >> serial_no = 93
> >> primary = 192.168.3.28
> >> service = lvs
> >> backup = 0.0.0.0
> >> heartbeat = 1
> >> heartbeat_port = 539
> >> keepalive = 6
> >> deadtime = 18
> >> network = direct
> >> debug_level = NONE
> >> virtual test1 {
> >> active = 1
> >> address = 192.168.3.40 eth0:1
> >> vip_nmask = 255.255.248.0
> >> port = 80
> >> expect = "HTTP"
> >> use_regex = 0
> >> load_monitor = none
> >> scheduler = lc
> >> protocol = tcp
> >> timeout = 6
> >> reentry = 15
> >> quiesce_server = 0
> >> server Speedy {
> >> address = 192.168.18.29
> >> active = 0
> >> port = 80
> >> weight = 1
> >> }
> >> server test1 {
> >> address = 65.39.179.197
> >> active = 0
> >> port = 80
> >> weight = 1
> >> }
> >>
> >>
> >>
> >> Mike
> >>
> >>
> >> On 03/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
> >> > Hello Mike,
> >> >
> >> > Thanks for providing helpful information.
> >> >
> >> > Now as i understood from your configuration, you have two networks on
> >> eth0.
> >> >
> >> > 1. 192.168.3.x (on eth0)
> >> > 2. 192.168.0.x (on eth0:1).. (Is it map with any external ip address?)
> >> > (Please provide netstat -rn output here.)
> >> >
> >> > Your one of real server is on completely new network subnet
> >> (192.168.18.x)
> >> > and your second real server is on public ip 65.39.179.197 and
> currently
> >> > none
> >> > of them are active.
> >> >
> >> > I don't think this configuration will work.
> >> >
> >> > Your configuration should be like:
> >> >
> >> > 1. Any network like 192.168.0.x on (eth0)
> >> > 2. Floating IP Address/Virtual IP Address 192.168.0.254 on (eth0:1)
> and
> >> it
> >> > must be map with any pubic ip address in case you want to access this
> >> > VIP
> >> > from outside of your network. During testing it is not required to map
> >> > it
> >> > with any public ip address.
> >> > 3. Your real server should be on the same network 192.168.0.x (I.e.
> >> > 192.168.0.1/2/3).
> >> >
> >> > If you are planning to use Linux Director in Direct Mode then their
> >> > must
> >> be
> >> > an existing gateway available.
> >> >
> >> > All real servers and Linux Director should point to their gateway
> >> > towards
> >> > that router/gateway.
> >> >
> >> > As far as your listing of port/service question is concern, If your
> >> > linux
> >> > director works properly, still your linux director will not listen on
> >> port
> >> > 80 but your load balancing will work. (I was also confused on this for
> >> > 2
> >> > days and after 2 days i realize that load balancing is working
> although
> >> > port
> >> > 80 is not listing.)
> >> >
> >> > Note: Please stick with one configuration as when you post your
> >> > problem,
> >> > your linux directory was working under Direct mode and now it is
> >> > working
> >> in
> >> > tunnel mode. (I have no experience of tunnel mode but i can help you
> on
> >> > direct and nat mode.)
> >> >
> >> > Regards
> >> > Tapan Thapa
> >> > India
> >> >
> >> > On Wed, Nov 4, 2009 at 1:16 AM, mojorising <moj0rising at aim.com>
> wrote:
> >> >
> >> >> Thanks for your offers of help!
> >> >>
> >> >> I have made some changes since reading your message saying the
> servers
> >> >> should all be on the same net -- now I have one real server and it is
> >> >> on the same network as the load balancer. The output of the ipvsadmn
> >> >> command you requested is below.
> >> >>
> >> >> [root at omsbuild ~]# ipvsadm -L -n
> >> >> IP Virtual Server version 1.2.1 (size=4096)
> >> >> Prot LocalAddress:Port Scheduler Flags
> >> >> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> >> >> TCP 192.168.0.69:80 lc
> >> >>
> >> >> NIC/IP information:
> >> >>
> >> >> [root at omsbuild ~]# ifconfig -a
> >> >> eth0 Link encap:Ethernet HWaddr 00:50:56:AE:14:E3
> >> >> inet addr:192.168.3.28 Bcast:192.168.7.255
> >> >> Mask:255.255.248.0
> >> >> inet6 addr: fe80::250:56ff:feae:14e3/64 Scope:Link
> >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> >> RX packets:35121740 errors:1120 dropped:1231 overruns:0
> >> >> frame:0
> >> >> TX packets:8682408 errors:0 dropped:0 overruns:0 carrier:0
> >> >> collisions:0 txqueuelen:1000
> >> >> RX bytes:4182471094 (3.8 GiB) TX bytes:606337720 (578.2
> MiB)
> >> >> Interrupt:177 Base address:0x1400
> >> >>
> >> >> eth0:1 Link encap:Ethernet HWaddr 00:50:56:AE:14:E3
> >> >> inet addr:192.168.0.69 Bcast:192.168.7.255
> >> >> Mask:255.255.248.0
> >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >> >> Interrupt:177 Base address:0x1400
> >> >>
> >> >>
> >> >> I'm still getting the same errors from nanny even though the real
> >> >> server is now on the same net as the load balancer:
> >> >>
> >> >> Nov 3 10:44:22 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed
> >> out
> >> >>
> >> >> As represented in eth0:1, my virtual server is listening on
> >> >> 192.168.0.69:80. If I do netstat, do a port/service check from a
> >> >> workstation to that IP or similar, shouldn't I see the load balancer
> >> >> listening on 80? Right now, I do not see the load balancer waiting
> for
> >> >> connections on port 80.
> >> >>
> >> >> Here is what my lvs.conf file looks like now:
> >> >>
> >> >> serial_no = 89
> >> >> primary = 192.168.3.28
> >> >> service = lvs
> >> >> backup = 0.0.0.0
> >> >> heartbeat = 1
> >> >> heartbeat_port = 539
> >> >> keepalive = 6
> >> >> deadtime = 18
> >> >> network = tunnel
> >> >> debug_level = NONE
> >> >> virtual test1 {
> >> >> active = 1
> >> >> address = 192.168.0.69 eth0:1
> >> >> vip_nmask = 255.255.248.0
> >> >> port = 80
> >> >> expect = "HTTP"
> >> >> use_regex = 0
> >> >> load_monitor = none
> >> >> scheduler = lc
> >> >> protocol = tcp
> >> >> timeout = 6
> >> >> reentry = 15
> >> >> quiesce_server = 0
> >> >> server Speedy {
> >> >> address = 192.168.18.29
> >> >> active = 0
> >> >> port = 80
> >> >> weight = 1
> >> >> }
> >> >> server test1 {
> >> >> address = 65.39.179.197
> >> >> active = 0
> >> >> port = 80
> >> >> weight = 1
> >> >> }
> >> >> :
> >> >>
> >> >>
> >> >> Mike
> >> >>
> >> >>
> >> >> On 02/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
> >> >> > Hello Mike,
> >> >> >
> >> >> > I am not an expert in IPVS but recently i have setup IPVS with the
> >> help
> >> >> of
> >> >> > Piranha and i am quite comfortable with IPVS.
> >> >> >
> >> >> > Please let me know your Network diagram and also the output of
> below
> >> >> > mentioned command.
> >> >> >
> >> >> > ipvsadm -L -n
> >> >> >
> >> >> > ----------------------------------------
> >> >> > I think your network diagram should be---
> >> >> >
> >> >> > Linux Director ----(One NIC)--->First Real Server (One NIC)
> >> >> > --->Second Real Server (One
> >> NIC)
> >> >> > ----------------------------------------------
> >> >> >
> >> >> > Your Linux Director and Your real server should be on the same
> >> >> > network
> >> >> > segment. Please also post your Linux Director's (Where you have
> >> >> > installed
> >> >> > Piranha) Network cards ip information.
> >> >> >
> >> >> >
> >> >> > Regards
> >> >> > Tapan Thapa
> >> >> > India
> >> >> >
> >> >> > On Tue, Nov 3, 2009 at 6:09 AM, mojorising <moj0rising at aim.com>
> >> wrote:
> >> >> >
> >> >> >> Hello!
> >> >> >>
> >> >> >> I have set up a test load balancer with IPVS and Piranha-GUI. For
> >> some
> >> >> >> reason, when I attempt to connect to one of the two web servers I
> >> have
> >> >> >> set-up via the load balancer's virtual IP, the load balancer does
> >> >> >> not
> >> >> >> seem to pass those requests on to the real servers.
> >> >> >>
> >> >> >> The firewall on the Piranha box is off and I can successfully
> >> >> >> establish HTTP sessions with netcat and telnet from the Piranha
> box
> >> as
> >> >> >> well as from my workstation. So the web services are running and
> >> >> >> connectivity to them is good.
> >> >> >>
> >> >> >> The error I'm getting in /var/log/messages is (public IP changed
> >> >> >> for
> >> >> >> privacy):
> >> >> >>
> >> >> >> Nov 2 14:28:09 omsbuild nanny[13583]: READ to 65.39.169.xxx:80
> >> >> >> timed
> >> >> out
> >> >> >> Nov 2 14:28:10 omsbuild nanny[13582]: READ to
> >> >> >> 192.168.18.29:80timed
> >> >> out
> >> >> >>
> >> >> >> It looks like nanny can't talk to the web servers but I can't
> >> >> >> figure
> >> >> >> out why. That may not be the only problem I have here but it's
> >> >> >> probably one of them. All the other services are up and seem to be
> >> >> >> running fine.
> >> >> >>
> >> >> >> I've googled around quite a bit and checked the documentation but
> I
> >> >> >> haven't found anything in those places that gets me to a solution.
> >> >> >>
> >> >> >> Can anyone out there give me a little push in the right direction
> >> >> >> as
> >> >> >> to what the problem might be?
> >> >> >>
> >> >> >>
> >> >> >> Thank you!
> >> >> >>
> >> >> >> Mike
> >> >> >>
> >> >> >>
> >> >> >> My lvs.conf file:
> >> >> >>
> >> >> >> serial_no = 76
> >> >> >> primary = 192.168.3.28
> >> >> >> service = lvs
> >> >> >> backup = 0.0.0.0
> >> >> >> heartbeat = 1
> >> >> >> heartbeat_port = 539
> >> >> >> keepalive = 6
> >> >> >> deadtime = 18
> >> >> >> network = direct
> >> >> >> debug_level = NONE
> >> >> >> virtual test1 {
> >> >> >> active = 1
> >> >> >> address = 192.168.0.69 eth0:1
> >> >> >> vip_nmask = 255.255.248.0
> >> >> >> port = 3128
> >> >> >> expect = "HTTP"
> >> >> >> use_regex = 0
> >> >> >> load_monitor = none
> >> >> >> scheduler = lc
> >> >> >> protocol = tcp
> >> >> >> timeout = 6
> >> >> >> reentry = 15
> >> >> >> quiesce_server = 0
> >> >> >> server Speedy {
> >> >> >> address = 192.168.18.29
> >> >> >> active = 1
> >> >> >> port = 80
> >> >> >> weight = 1
> >> >> >> }
> >> >> >> server test1 {
> >> >> >> address = 65.39.169.xxx
> >> >> >> active = 1
> >> >> >> port = 80
> >> >> >> weight = 1
> >> >> >> }
> >> >> >>
> >> >> >> _______________________________________________
> >> >> >> Piranha-list mailing list
> >> >> >> Piranha-list at redhat.com
> >> >> >> https://www.redhat.com/mailman/listinfo/piranha-list
> >> >> >>
> >> >> >
> >> >>
> >> >> _______________________________________________
> >> >> Piranha-list mailing list
> >> >> Piranha-list at redhat.com
> >> >> https://www.redhat.com/mailman/listinfo/piranha-list
> >> >>
> >> >
> >>
> >> _______________________________________________
> >> Piranha-list mailing list
> >> Piranha-list at redhat.com
> >> https://www.redhat.com/mailman/listinfo/piranha-list
> >>
> >
>
> _______________________________________________
> Piranha-list mailing list
> Piranha-list at redhat.com
> https://www.redhat.com/mailman/listinfo/piranha-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/piranha-list/attachments/20091106/6f5cda5a/attachment.htm>
More information about the Piranha-list
mailing list