nanny "Read Timed Out" Errors

Tapan Thapa tapan.thapa2000 at gmail.com
Thu Nov 5 03:46:08 UTC 2009


Hello Mike,

Now your network status looks good.

But still i can not see any real server on the same network. i.e.
192.168.3.x.

As per your lvs.cf, you have configured two real servers. First one is
server Speedy and second one is server test1 and currently both are not
active. (active = 0) They should be (active = 1).

And also your real servers are not in right subnet.

Your real server should be on same 192.168.3.x network.

Your example lvs.cf should look like:

serial_no = 93
primary = 192.168.3.28
service = lvs
backup = 0.0.0.0
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 18
network = direct
debug_level = NONE
virtual test1 {
    active = 1
     address = 192.168.3.40 eth0:1
    vip_nmask = 255.255.248.0
    port = 80
    expect = "HTTP"
    use_regex = 0
    load_monitor = none
    scheduler = lc
    protocol = tcp
    timeout = 6
    reentry = 15
    quiesce_server = 0
    server Speedy {
        address = 192.168.3.29
        active = 1
        port = 80
        weight = 1
    }
    server test1 {
        address = 192.168.3.30
        active = 1
        port = 80
        weight = 1
    }

Please change the ip address of Speedy server to 192.168.3.29 and test1
server to 192.168.3.30 with subnet mask of 255.255.248.0 and restart network
and httpd service.

then fire below mentioned commands in both real servers (Not in lvs server):

chkconfig iptables on
iptables -F
iptables -t nat -A PREROUTING -p tcp --dport 80 -d 192.168.3.40 -j REDIRECT
service iptables save

then please restart pulse service at linux director server (lvs server) and
wait for 2 minutes. and then check the status of ipvsadm -L -n command and
let me know in case any issues.


Regards
Tapan Thapa
India



On Thu, Nov 5, 2009 at 1:25 AM, mojorising <moj0rising at aim.com> wrote:

> Tapan, sorry for confusing you. I overlooked my virtual IP and
> accidedntally left it with an IP on the wrong net. This is now
> corrected.
>
> Those real servers on other nets are still in my configuration but
> they are "down," as they were before. I do have one real server up on
> the proper net --       192.168.3.38.
>
>
> My present network interface set-up:
>
> eth0      Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
>          inet addr:192.168.3.28  Bcast:192.168.7.255  Mask:255.255.248.0
>          inet6 addr: fe80::250:56ff:feae:14e3/64 Scope:Link
>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:36180876 errors:1122 dropped:1234 overruns:0 frame:0
>          TX packets:8729361 errors:0 dropped:0 overruns:0 carrier:0
>          collisions:0 txqueuelen:1000
>          RX bytes:72196093 (68.8 MiB)  TX bytes:610192805 (581.9 MiB)
>           Interrupt:177 Base address:0x1400
>
> eth0:1    Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
>           inet addr:192.168.3.40  Bcast:192.168.7.255  Mask:255.255.248.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>          Interrupt:177 Base address:0x1400
>
>
> A problem is that I am still getting those error messages from nanny:
>
> Nov  4 11:06:52 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed out
> Nov  4 11:06:53 omsbuild nanny[20548]: READ to 192.168.3.38:80 timed out
> Nov  4 11:07:04 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed out
> Nov  4 11:07:05 omsbuild nanny[20548]: READ to 192.168.3.38:80 timed out
>
>
> My lvs.cf file now:
>
> serial_no = 93
> primary = 192.168.3.28
> service = lvs
> backup = 0.0.0.0
> heartbeat = 1
> heartbeat_port = 539
> keepalive = 6
> deadtime = 18
> network = direct
> debug_level = NONE
> virtual test1 {
>     active = 1
>      address = 192.168.3.40 eth0:1
>      vip_nmask = 255.255.248.0
>     port = 80
>     expect = "HTTP"
>     use_regex = 0
>     load_monitor = none
>     scheduler = lc
>     protocol = tcp
>     timeout = 6
>     reentry = 15
>     quiesce_server = 0
>     server Speedy {
>         address = 192.168.18.29
>         active = 0
>         port = 80
>         weight = 1
>     }
>     server test1 {
>         address = 65.39.179.197
>         active = 0
>         port = 80
>         weight = 1
>     }
>
>
>
> Mike
>
>
> On 03/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
> > Hello Mike,
> >
> > Thanks for providing helpful information.
> >
> > Now as i understood from your configuration, you have two networks on
> eth0.
> >
> > 1. 192.168.3.x (on eth0)
> > 2. 192.168.0.x (on eth0:1).. (Is it map with any external ip address?)
> > (Please provide netstat -rn output here.)
> >
> > Your one of real server is on completely new network subnet
> (192.168.18.x)
> > and your second real server is on public ip 65.39.179.197 and currently
> > none
> > of them are active.
> >
> > I don't think this configuration will work.
> >
> > Your configuration should be like:
> >
> > 1. Any network like 192.168.0.x on (eth0)
> > 2. Floating IP Address/Virtual IP Address 192.168.0.254 on (eth0:1) and
> it
> > must be map with any pubic ip address in case you want to access this VIP
> > from outside of your network. During testing it is not required to map it
> > with any public ip address.
> > 3. Your real server should be on the same network 192.168.0.x (I.e.
> > 192.168.0.1/2/3).
> >
> > If you are planning to use Linux Director in Direct Mode then their must
> be
> > an existing gateway available.
> >
> > All real servers and Linux Director should point to their gateway towards
> > that router/gateway.
> >
> > As far as your listing of port/service question is concern, If your linux
> > director works properly, still your linux director will not listen on
> port
> > 80 but your load balancing will work. (I was also confused on this for 2
> > days and after 2 days i realize that load balancing is working although
> > port
> > 80 is not listing.)
> >
> > Note: Please stick with one configuration as when you post your problem,
> > your linux directory was working under Direct mode and now it is working
> in
> > tunnel mode. (I have no experience of tunnel mode but i can help you on
> > direct and nat mode.)
> >
> > Regards
> > Tapan Thapa
> > India
> >
> > On Wed, Nov 4, 2009 at 1:16 AM, mojorising <moj0rising at aim.com> wrote:
> >
> >> Thanks for your offers of help!
> >>
> >> I have made some changes since reading your message saying the servers
> >> should all be on the same net -- now I have one real server and it is
> >> on the same network as the load balancer. The output of the ipvsadmn
> >> command you requested is below.
> >>
> >> [root at omsbuild ~]# ipvsadm -L -n
> >> IP Virtual Server version 1.2.1 (size=4096)
> >> Prot LocalAddress:Port Scheduler Flags
> >>  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
> >> TCP  192.168.0.69:80 lc
> >>
> >> NIC/IP information:
> >>
> >> [root at omsbuild ~]# ifconfig -a
> >> eth0      Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
> >>          inet addr:192.168.3.28  Bcast:192.168.7.255  Mask:255.255.248.0
> >>          inet6 addr: fe80::250:56ff:feae:14e3/64 Scope:Link
> >>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>          RX packets:35121740 errors:1120 dropped:1231 overruns:0 frame:0
> >>          TX packets:8682408 errors:0 dropped:0 overruns:0 carrier:0
> >>          collisions:0 txqueuelen:1000
> >>          RX bytes:4182471094 (3.8 GiB)  TX bytes:606337720 (578.2 MiB)
> >>          Interrupt:177 Base address:0x1400
> >>
> >> eth0:1    Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
> >>          inet addr:192.168.0.69  Bcast:192.168.7.255  Mask:255.255.248.0
> >>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>          Interrupt:177 Base address:0x1400
> >>
> >>
> >> I'm still getting the same errors from nanny even though the real
> >> server is now on the same net as the load balancer:
> >>
> >> Nov  3 10:44:22 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed
> out
> >>
> >> As represented in eth0:1, my virtual server is listening on
> >> 192.168.0.69:80. If I do netstat, do a port/service check from a
> >> workstation to that IP or similar, shouldn't I see the load balancer
> >> listening on 80? Right now, I do not see the load balancer waiting for
> >> connections on port 80.
> >>
> >> Here is what my lvs.conf file looks like now:
> >>
> >> serial_no = 89
> >> primary = 192.168.3.28
> >> service = lvs
> >> backup = 0.0.0.0
> >> heartbeat = 1
> >> heartbeat_port = 539
> >> keepalive = 6
> >> deadtime = 18
> >> network = tunnel
> >> debug_level = NONE
> >> virtual test1 {
> >>     active = 1
> >>     address = 192.168.0.69 eth0:1
> >>     vip_nmask = 255.255.248.0
> >>      port = 80
> >>      expect = "HTTP"
> >>     use_regex = 0
> >>     load_monitor = none
> >>     scheduler = lc
> >>     protocol = tcp
> >>     timeout = 6
> >>     reentry = 15
> >>     quiesce_server = 0
> >>     server Speedy {
> >>         address = 192.168.18.29
> >>         active = 0
> >>         port = 80
> >>         weight = 1
> >>     }
> >>     server test1 {
> >>          address = 65.39.179.197
> >>         active = 0
> >>         port = 80
> >>         weight = 1
> >>     }
> >> :
> >>
> >>
> >> Mike
> >>
> >>
> >> On 02/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
> >> > Hello Mike,
> >> >
> >> > I am not an expert in IPVS but recently i have setup IPVS with the
> help
> >> of
> >> > Piranha and i am quite comfortable with IPVS.
> >> >
> >> > Please let me know your Network diagram and also the output of below
> >> > mentioned command.
> >> >
> >> > ipvsadm -L -n
> >> >
> >> > ----------------------------------------
> >> > I think your network diagram should be---
> >> >
> >> > Linux Director ----(One NIC)--->First Real Server (One NIC)
> >> >                                        --->Second Real Server (One
> NIC)
> >> > ----------------------------------------------
> >> >
> >> > Your Linux Director and Your real server should be on the same network
> >> > segment. Please also post your Linux Director's (Where you have
> >> > installed
> >> > Piranha) Network cards ip information.
> >> >
> >> >
> >> > Regards
> >> > Tapan Thapa
> >> > India
> >> >
> >> > On Tue, Nov 3, 2009 at 6:09 AM, mojorising <moj0rising at aim.com>
> wrote:
> >> >
> >> >> Hello!
> >> >>
> >> >> I have set up a test load balancer with IPVS and Piranha-GUI. For
> some
> >> >> reason, when I attempt to connect to one of the two web servers I
> have
> >> >> set-up via the load balancer's virtual IP, the load balancer does not
> >> >> seem to pass those requests on to the real servers.
> >> >>
> >> >> The firewall on the Piranha box is off and I can successfully
> >> >> establish HTTP sessions with netcat and telnet from the Piranha box
> as
> >> >> well as from my workstation. So the web services are running and
> >> >> connectivity to them is good.
> >> >>
> >> >> The error I'm getting in /var/log/messages is (public IP changed for
> >> >> privacy):
> >> >>
> >> >> Nov  2 14:28:09 omsbuild nanny[13583]: READ to 65.39.169.xxx:80 timed
> >> out
> >> >> Nov  2 14:28:10 omsbuild nanny[13582]: READ to 192.168.18.29:80timed
> >> out
> >> >>
> >> >> It looks like nanny can't talk to the web servers but I can't figure
> >> >> out why. That may not be the only problem I have here but it's
> >> >> probably one of them. All the other services are up and seem to be
> >> >> running fine.
> >> >>
> >> >> I've googled around quite a bit and checked the documentation but I
> >> >> haven't found anything in those places that gets me to a solution.
> >> >>
> >> >> Can anyone out there give me a little push in the right direction as
> >> >> to what the problem might be?
> >> >>
> >> >>
> >> >> Thank you!
> >> >>
> >> >> Mike
> >> >>
> >> >>
> >> >> My lvs.conf file:
> >> >>
> >> >> serial_no = 76
> >> >> primary = 192.168.3.28
> >> >> service = lvs
> >> >> backup = 0.0.0.0
> >> >> heartbeat = 1
> >> >> heartbeat_port = 539
> >> >> keepalive = 6
> >> >> deadtime = 18
> >> >> network = direct
> >> >> debug_level = NONE
> >> >> virtual test1 {
> >> >>     active = 1
> >> >>     address = 192.168.0.69 eth0:1
> >> >>     vip_nmask = 255.255.248.0
> >> >>     port = 3128
> >> >>     expect = "HTTP"
> >> >>     use_regex = 0
> >> >>     load_monitor = none
> >> >>     scheduler = lc
> >> >>     protocol = tcp
> >> >>     timeout = 6
> >> >>     reentry = 15
> >> >>     quiesce_server = 0
> >> >>     server Speedy {
> >> >>         address = 192.168.18.29
> >> >>         active = 1
> >> >>         port = 80
> >> >>         weight = 1
> >> >>     }
> >> >>     server test1 {
> >> >>         address = 65.39.169.xxx
> >> >>         active = 1
> >> >>         port = 80
> >> >>         weight = 1
> >> >>     }
> >> >>
> >> >> _______________________________________________
> >> >> Piranha-list mailing list
> >> >> Piranha-list at redhat.com
> >> >> https://www.redhat.com/mailman/listinfo/piranha-list
> >> >>
> >> >
> >>
> >> _______________________________________________
> >> Piranha-list mailing list
> >> Piranha-list at redhat.com
> >> https://www.redhat.com/mailman/listinfo/piranha-list
> >>
> >
>
> _______________________________________________
> Piranha-list mailing list
> Piranha-list at redhat.com
> https://www.redhat.com/mailman/listinfo/piranha-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/piranha-list/attachments/20091105/82a316c0/attachment.htm>


More information about the Piranha-list mailing list