nanny "Read Timed Out" Errors

mojorising moj0rising at aim.com
Thu Nov 5 18:53:16 UTC 2009


Okay. It seems I missed a critical piece of my config file when I
copy/pasted it to you. Sorry about that.

So here is my ha.cf file now

[root at omsbuild ~]# cat /etc/sysconfig/ha/lvs.cf
serial_no = 93
primary = 192.168.3.28
service = lvs
backup = 0.0.0.0
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 18
network = direct
debug_level = NONE
virtual test1 {
     active = 1
     address = 192.168.3.40 eth0:1
     vip_nmask = 255.255.248.0
     port = 80
     expect = "HTTP"
     use_regex = 0
     load_monitor = none
     scheduler = lc
     protocol = tcp
     timeout = 6
     reentry = 15
     quiesce_server = 0
     server kiwidev4 {
         address = 192.168.3.38
         active = 1
         port = 80
         weight = 1
     }
}


I took out those other machines because I can not change their IPs
(I'm just using them for testing). So in their place, I put a machine
(kiwidev4) that happens to be on the same subnet as the LVS box.
kiwidev4 was always there and active but that part of the config file
was accidentally clipped off from my message.  :(

I can not change those iptables rules at this time because that
kiwidev4 box may be in use for some other testing at the moment. Can
we do this without making the specified changes to iptables? It seems
we shouldn't need to do that. I will eventually be using LVS to
balance traffic to Windows machines as well so I need to be able to do
without iptables for that reason also.


Mike


On 04/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
> Hello Mike,
>
> Now your network status looks good.
>
> But still i can not see any real server on the same network. i.e.
> 192.168.3.x.
>
> As per your lvs.cf, you have configured two real servers. First one is
> server Speedy and second one is server test1 and currently both are not
> active. (active = 0) They should be (active = 1).
>
> And also your real servers are not in right subnet.
>
> Your real server should be on same 192.168.3.x network.
>
> Your example lvs.cf should look like:
>
> serial_no = 93
> primary = 192.168.3.28
> service = lvs
> backup = 0.0.0.0
> heartbeat = 1
> heartbeat_port = 539
> keepalive = 6
> deadtime = 18
> network = direct
> debug_level = NONE
> virtual test1 {
>     active = 1
>      address = 192.168.3.40 eth0:1
>     vip_nmask = 255.255.248.0
>     port = 80
>     expect = "HTTP"
>     use_regex = 0
>     load_monitor = none
>     scheduler = lc
>     protocol = tcp
>     timeout = 6
>     reentry = 15
>     quiesce_server = 0
>     server Speedy {
>         address = 192.168.3.29
>         active = 1
>         port = 80
>         weight = 1
>     }
>     server test1 {
>         address = 192.168.3.30
>         active = 1
>         port = 80
>         weight = 1
>     }
>
> Please change the ip address of Speedy server to 192.168.3.29 and test1
> server to 192.168.3.30 with subnet mask of 255.255.248.0 and restart
> network
> and httpd service.
>
> then fire below mentioned commands in both real servers (Not in lvs
> server):
>
> chkconfig iptables on
> iptables -F
> iptables -t nat -A PREROUTING -p tcp --dport 80 -d 192.168.3.40 -j REDIRECT
> service iptables save
>
> then please restart pulse service at linux director server (lvs server) and
> wait for 2 minutes. and then check the status of ipvsadm -L -n command and
> let me know in case any issues.
>
>
> Regards
> Tapan Thapa
> India
>
>
>
> On Thu, Nov 5, 2009 at 1:25 AM, mojorising <moj0rising at aim.com> wrote:
>
>> Tapan, sorry for confusing you. I overlooked my virtual IP and
>> accidedntally left it with an IP on the wrong net. This is now
>> corrected.
>>
>> Those real servers on other nets are still in my configuration but
>> they are "down," as they were before. I do have one real server up on
>> the proper net --       192.168.3.38.
>>
>>
>> My present network interface set-up:
>>
>> eth0      Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
>>          inet addr:192.168.3.28  Bcast:192.168.7.255  Mask:255.255.248.0
>>          inet6 addr: fe80::250:56ff:feae:14e3/64 Scope:Link
>>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>           RX packets:36180876 errors:1122 dropped:1234 overruns:0 frame:0
>>          TX packets:8729361 errors:0 dropped:0 overruns:0 carrier:0
>>          collisions:0 txqueuelen:1000
>>          RX bytes:72196093 (68.8 MiB)  TX bytes:610192805 (581.9 MiB)
>>           Interrupt:177 Base address:0x1400
>>
>> eth0:1    Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
>>           inet addr:192.168.3.40  Bcast:192.168.7.255  Mask:255.255.248.0
>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>          Interrupt:177 Base address:0x1400
>>
>>
>> A problem is that I am still getting those error messages from nanny:
>>
>> Nov  4 11:06:52 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed out
>> Nov  4 11:06:53 omsbuild nanny[20548]: READ to 192.168.3.38:80 timed out
>> Nov  4 11:07:04 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed out
>> Nov  4 11:07:05 omsbuild nanny[20548]: READ to 192.168.3.38:80 timed out
>>
>>
>> My lvs.cf file now:
>>
>> serial_no = 93
>> primary = 192.168.3.28
>> service = lvs
>> backup = 0.0.0.0
>> heartbeat = 1
>> heartbeat_port = 539
>> keepalive = 6
>> deadtime = 18
>> network = direct
>> debug_level = NONE
>> virtual test1 {
>>     active = 1
>>      address = 192.168.3.40 eth0:1
>>      vip_nmask = 255.255.248.0
>>     port = 80
>>     expect = "HTTP"
>>     use_regex = 0
>>     load_monitor = none
>>     scheduler = lc
>>     protocol = tcp
>>     timeout = 6
>>     reentry = 15
>>     quiesce_server = 0
>>     server Speedy {
>>         address = 192.168.18.29
>>         active = 0
>>         port = 80
>>         weight = 1
>>     }
>>     server test1 {
>>         address = 65.39.179.197
>>         active = 0
>>         port = 80
>>         weight = 1
>>     }
>>
>>
>>
>> Mike
>>
>>
>> On 03/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
>> > Hello Mike,
>> >
>> > Thanks for providing helpful information.
>> >
>> > Now as i understood from your configuration, you have two networks on
>> eth0.
>> >
>> > 1. 192.168.3.x (on eth0)
>> > 2. 192.168.0.x (on eth0:1).. (Is it map with any external ip address?)
>> > (Please provide netstat -rn output here.)
>> >
>> > Your one of real server is on completely new network subnet
>> (192.168.18.x)
>> > and your second real server is on public ip 65.39.179.197 and currently
>> > none
>> > of them are active.
>> >
>> > I don't think this configuration will work.
>> >
>> > Your configuration should be like:
>> >
>> > 1. Any network like 192.168.0.x on (eth0)
>> > 2. Floating IP Address/Virtual IP Address 192.168.0.254 on (eth0:1) and
>> it
>> > must be map with any pubic ip address in case you want to access this
>> > VIP
>> > from outside of your network. During testing it is not required to map
>> > it
>> > with any public ip address.
>> > 3. Your real server should be on the same network 192.168.0.x (I.e.
>> > 192.168.0.1/2/3).
>> >
>> > If you are planning to use Linux Director in Direct Mode then their
>> > must
>> be
>> > an existing gateway available.
>> >
>> > All real servers and Linux Director should point to their gateway
>> > towards
>> > that router/gateway.
>> >
>> > As far as your listing of port/service question is concern, If your
>> > linux
>> > director works properly, still your linux director will not listen on
>> port
>> > 80 but your load balancing will work. (I was also confused on this for
>> > 2
>> > days and after 2 days i realize that load balancing is working although
>> > port
>> > 80 is not listing.)
>> >
>> > Note: Please stick with one configuration as when you post your
>> > problem,
>> > your linux directory was working under Direct mode and now it is
>> > working
>> in
>> > tunnel mode. (I have no experience of tunnel mode but i can help you on
>> > direct and nat mode.)
>> >
>> > Regards
>> > Tapan Thapa
>> > India
>> >
>> > On Wed, Nov 4, 2009 at 1:16 AM, mojorising <moj0rising at aim.com> wrote:
>> >
>> >> Thanks for your offers of help!
>> >>
>> >> I have made some changes since reading your message saying the servers
>> >> should all be on the same net -- now I have one real server and it is
>> >> on the same network as the load balancer. The output of the ipvsadmn
>> >> command you requested is below.
>> >>
>> >> [root at omsbuild ~]# ipvsadm -L -n
>> >> IP Virtual Server version 1.2.1 (size=4096)
>> >> Prot LocalAddress:Port Scheduler Flags
>> >>  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
>> >> TCP  192.168.0.69:80 lc
>> >>
>> >> NIC/IP information:
>> >>
>> >> [root at omsbuild ~]# ifconfig -a
>> >> eth0      Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
>> >>          inet addr:192.168.3.28  Bcast:192.168.7.255
>> >> Mask:255.255.248.0
>> >>          inet6 addr: fe80::250:56ff:feae:14e3/64 Scope:Link
>> >>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> >>          RX packets:35121740 errors:1120 dropped:1231 overruns:0
>> >> frame:0
>> >>          TX packets:8682408 errors:0 dropped:0 overruns:0 carrier:0
>> >>          collisions:0 txqueuelen:1000
>> >>          RX bytes:4182471094 (3.8 GiB)  TX bytes:606337720 (578.2 MiB)
>> >>          Interrupt:177 Base address:0x1400
>> >>
>> >> eth0:1    Link encap:Ethernet  HWaddr 00:50:56:AE:14:E3
>> >>          inet addr:192.168.0.69  Bcast:192.168.7.255
>> >> Mask:255.255.248.0
>> >>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>> >>          Interrupt:177 Base address:0x1400
>> >>
>> >>
>> >> I'm still getting the same errors from nanny even though the real
>> >> server is now on the same net as the load balancer:
>> >>
>> >> Nov  3 10:44:22 omsbuild nanny[16919]: READ to 192.168.3.38:80 timed
>> out
>> >>
>> >> As represented in eth0:1, my virtual server is listening on
>> >> 192.168.0.69:80. If I do netstat, do a port/service check from a
>> >> workstation to that IP or similar, shouldn't I see the load balancer
>> >> listening on 80? Right now, I do not see the load balancer waiting for
>> >> connections on port 80.
>> >>
>> >> Here is what my lvs.conf file looks like now:
>> >>
>> >> serial_no = 89
>> >> primary = 192.168.3.28
>> >> service = lvs
>> >> backup = 0.0.0.0
>> >> heartbeat = 1
>> >> heartbeat_port = 539
>> >> keepalive = 6
>> >> deadtime = 18
>> >> network = tunnel
>> >> debug_level = NONE
>> >> virtual test1 {
>> >>     active = 1
>> >>     address = 192.168.0.69 eth0:1
>> >>     vip_nmask = 255.255.248.0
>> >>      port = 80
>> >>      expect = "HTTP"
>> >>     use_regex = 0
>> >>     load_monitor = none
>> >>     scheduler = lc
>> >>     protocol = tcp
>> >>     timeout = 6
>> >>     reentry = 15
>> >>     quiesce_server = 0
>> >>     server Speedy {
>> >>         address = 192.168.18.29
>> >>         active = 0
>> >>         port = 80
>> >>         weight = 1
>> >>     }
>> >>     server test1 {
>> >>          address = 65.39.179.197
>> >>         active = 0
>> >>         port = 80
>> >>         weight = 1
>> >>     }
>> >> :
>> >>
>> >>
>> >> Mike
>> >>
>> >>
>> >> On 02/11/2009, Tapan Thapa <tapan.thapa2000 at gmail.com> wrote:
>> >> > Hello Mike,
>> >> >
>> >> > I am not an expert in IPVS but recently i have setup IPVS with the
>> help
>> >> of
>> >> > Piranha and i am quite comfortable with IPVS.
>> >> >
>> >> > Please let me know your Network diagram and also the output of below
>> >> > mentioned command.
>> >> >
>> >> > ipvsadm -L -n
>> >> >
>> >> > ----------------------------------------
>> >> > I think your network diagram should be---
>> >> >
>> >> > Linux Director ----(One NIC)--->First Real Server (One NIC)
>> >> >                                        --->Second Real Server (One
>> NIC)
>> >> > ----------------------------------------------
>> >> >
>> >> > Your Linux Director and Your real server should be on the same
>> >> > network
>> >> > segment. Please also post your Linux Director's (Where you have
>> >> > installed
>> >> > Piranha) Network cards ip information.
>> >> >
>> >> >
>> >> > Regards
>> >> > Tapan Thapa
>> >> > India
>> >> >
>> >> > On Tue, Nov 3, 2009 at 6:09 AM, mojorising <moj0rising at aim.com>
>> wrote:
>> >> >
>> >> >> Hello!
>> >> >>
>> >> >> I have set up a test load balancer with IPVS and Piranha-GUI. For
>> some
>> >> >> reason, when I attempt to connect to one of the two web servers I
>> have
>> >> >> set-up via the load balancer's virtual IP, the load balancer does
>> >> >> not
>> >> >> seem to pass those requests on to the real servers.
>> >> >>
>> >> >> The firewall on the Piranha box is off and I can successfully
>> >> >> establish HTTP sessions with netcat and telnet from the Piranha box
>> as
>> >> >> well as from my workstation. So the web services are running and
>> >> >> connectivity to them is good.
>> >> >>
>> >> >> The error I'm getting in /var/log/messages is (public IP changed
>> >> >> for
>> >> >> privacy):
>> >> >>
>> >> >> Nov  2 14:28:09 omsbuild nanny[13583]: READ to 65.39.169.xxx:80
>> >> >> timed
>> >> out
>> >> >> Nov  2 14:28:10 omsbuild nanny[13582]: READ to
>> >> >> 192.168.18.29:80timed
>> >> out
>> >> >>
>> >> >> It looks like nanny can't talk to the web servers but I can't
>> >> >> figure
>> >> >> out why. That may not be the only problem I have here but it's
>> >> >> probably one of them. All the other services are up and seem to be
>> >> >> running fine.
>> >> >>
>> >> >> I've googled around quite a bit and checked the documentation but I
>> >> >> haven't found anything in those places that gets me to a solution.
>> >> >>
>> >> >> Can anyone out there give me a little push in the right direction
>> >> >> as
>> >> >> to what the problem might be?
>> >> >>
>> >> >>
>> >> >> Thank you!
>> >> >>
>> >> >> Mike
>> >> >>
>> >> >>
>> >> >> My lvs.conf file:
>> >> >>
>> >> >> serial_no = 76
>> >> >> primary = 192.168.3.28
>> >> >> service = lvs
>> >> >> backup = 0.0.0.0
>> >> >> heartbeat = 1
>> >> >> heartbeat_port = 539
>> >> >> keepalive = 6
>> >> >> deadtime = 18
>> >> >> network = direct
>> >> >> debug_level = NONE
>> >> >> virtual test1 {
>> >> >>     active = 1
>> >> >>     address = 192.168.0.69 eth0:1
>> >> >>     vip_nmask = 255.255.248.0
>> >> >>     port = 3128
>> >> >>     expect = "HTTP"
>> >> >>     use_regex = 0
>> >> >>     load_monitor = none
>> >> >>     scheduler = lc
>> >> >>     protocol = tcp
>> >> >>     timeout = 6
>> >> >>     reentry = 15
>> >> >>     quiesce_server = 0
>> >> >>     server Speedy {
>> >> >>         address = 192.168.18.29
>> >> >>         active = 1
>> >> >>         port = 80
>> >> >>         weight = 1
>> >> >>     }
>> >> >>     server test1 {
>> >> >>         address = 65.39.169.xxx
>> >> >>         active = 1
>> >> >>         port = 80
>> >> >>         weight = 1
>> >> >>     }
>> >> >>
>> >> >> _______________________________________________
>> >> >> Piranha-list mailing list
>> >> >> Piranha-list at redhat.com
>> >> >> https://www.redhat.com/mailman/listinfo/piranha-list
>> >> >>
>> >> >
>> >>
>> >> _______________________________________________
>> >> Piranha-list mailing list
>> >> Piranha-list at redhat.com
>> >> https://www.redhat.com/mailman/listinfo/piranha-list
>> >>
>> >
>>
>> _______________________________________________
>> Piranha-list mailing list
>> Piranha-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/piranha-list
>>
>




More information about the Piranha-list mailing list