From brentonr at dorm.org Mon Oct 9 15:18:26 2006 From: brentonr at dorm.org (Brenton Rothchild) Date: Mon, 09 Oct 2006 10:18:26 -0500 Subject: [Cluster-devel] LVS port translation In-Reply-To: <452A5CD3.60500@cesca.es> References: <452A5CD3.60500@cesca.es> Message-ID: <452A6842.5020608@dorm.org> What version of piranha are you using? I have a patch against piranha-0.8.1 and piranha-0.8.2 that I've been using in heavy production for about 9 months now. We just added a "port" value to real server sections in the lvs.cf file, i.e. virtual server_XYZ { address = 123.456.789.1 eth0 active = 1 port = 80 ... (additional options) ... server node-X { address = 192.168.10.1 port = 82 active = 1 weight = 1 } ... } I can send the patch to the list if you're interested - shouldn't be too hard to patch against other versions (I haven't looked to see if the RPMs have been updated lately.) -Brenton Rothchild Jordi Prats wrote: > Hi all, > It's possible to configure piranha to allow to define a different port > on real servers using NAT? > > Using ipvsadm would be: > > /sbin/ipvsadm -a -t 123.456.789.1:80 -r 192.168.10.1:82 -m -w 1 > > On the current version is not possible, there is any patch to allow > this? If there is not, I could do it. Would it be accepted as a > contribution on main distribution? Witch roules would I have to follow? > > Thanks, > From brentonr at dorm.org Tue Oct 10 13:44:24 2006 From: brentonr at dorm.org (Brenton Rothchild) Date: Tue, 10 Oct 2006 08:44:24 -0500 Subject: [Linux-cluster] LVS port translation In-Reply-To: <452AA973.3010904@cesca.es> References: <452A5CD3.60500@cesca.es> <452A6842.5020608@dorm.org> <452AA973.3010904@cesca.es> Message-ID: <452BA3B8.6090009@dorm.org> Jordi Prats wrote: > Hi, > It would be great if you could send it to me! There's any reason why it > is not included in main distribution? > > If there's any thing left to do I could try to do it because I prefere > not to depen on a patch to upgrade. > > Thank you! > Jordi > Jordi, Attached is the patch I have been using. It does the following things: - Adds a "port" option to real-server blocks in lvs.cf. This value defaults to the virtual service port number if specified, and port 80 if neither are specified (this is the default as is currently in piranha). This port value is passed to the "-r" argument for ipvsadm as called by nanny. - Adds the "-r" argument to nanny itself, similar to the already existing "-p" argument. The manpage for nanny has been updated to reflect this usage. - The special token "%p" has been added to nanny's external check command syntax in order to utilize the specific real-server port number in command-line-specified external check commands with nanny. Also see nanny manpage updates. So basically what you can do is add a "port" value inside your real-server blocks in lvs.cf, and be on your way :) I don't know if there's anything left to do in order to have such a patch submitted to the piranha sources - I've never done that before. If anyone would like to get it in there, I would love it for the same reason: no more custom building/patching on upgrades :) Comments certainly welcome! -Brenton Rothchild -------------- next part -------------- A non-text attachment was scrubbed... Name: piranha-0.8.2-rport.patch Type: text/x-patch Size: 15270 bytes Desc: not available URL: From jprats at cesca.es Wed Oct 11 14:46:57 2006 From: jprats at cesca.es (Jordi Prats) Date: Wed, 11 Oct 2006 16:46:57 +0200 Subject: [Linux-cluster] LVS port translation In-Reply-To: <452BA3B8.6090009@dorm.org> References: <452A5CD3.60500@cesca.es> <452A6842.5020608@dorm.org> <452AA973.3010904@cesca.es> <452BA3B8.6090009@dorm.org> Message-ID: <452D03E1.4060904@cesca.es> Thank you very much Brenton! I'll add this to the PHP interface, this way this patch will be completly integrated to piranha. A general question to piranha-list: How can this patch be submited ? Jordi Brenton Rothchild wrote: > Jordi Prats wrote: >> Hi, >> It would be great if you could send it to me! There's any reason why it >> is not included in main distribution? >> >> If there's any thing left to do I could try to do it because I prefere >> not to depen on a patch to upgrade. >> >> Thank you! >> Jordi >> > > Jordi, > > Attached is the patch I have been using. > > It does the following things: > > - Adds a "port" option to real-server blocks in lvs.cf. This value > defaults > to the virtual service port number if specified, and port 80 if > neither are > specified (this is the default as is currently in piranha). This > port value > is passed to the "-r" argument for ipvsadm as called by nanny. > > - Adds the "-r" argument to nanny itself, similar to the already > existing "-p" > argument. The manpage for nanny has been updated to reflect this > usage. > > - The special token "%p" has been added to nanny's external check > command syntax > in order to utilize the specific real-server port number in > command-line-specified > external check commands with nanny. Also see nanny manpage updates. > > > So basically what you can do is add a "port" value inside your > real-server blocks > in lvs.cf, and be on your way :) > > I don't know if there's anything left to do in order to have such a > patch submitted > to the piranha sources - I've never done that before. If anyone would > like to get > it in there, I would love it for the same reason: no more custom > building/patching > on upgrades :) > > Comments certainly welcome! > > -Brenton Rothchild > > ------------------------------------------------------------------------ > > diff -uNr piranha-0.8.2-orig/lvsconfig.c piranha/lvsconfig.c > --- piranha-0.8.2-orig/lvsconfig.c 2006-01-12 15:54:59.000000000 -0600 > +++ piranha/lvsconfig.c 2006-10-10 08:18:01.000000000 -0500 > @@ -736,7 +736,9 @@ > OUTPUT_MISSING_NUM (LVS_SAW_AS_ISACTIVE, eservice->isActive, > -1, (char *) "\t\tactive = %d\n") > OUTPUT_MISSING_NUM (LVS_SAW_AS_WEIGHT, eservice->weight, > - 1, (char *) "\t\tweight = %d\n")} > + 1, (char *) "\t\tweight = %d\n") > + OUTPUT_MISSING_NUM (LVS_SAW_AS_PORT, eservice->port, > + 1, (char *) "\t\tport = %d\n")} > > > > @@ -782,6 +784,13 @@ > OUTPUT_NUM (eservice, weight, 2, LVS_SAW_AS_WEIGHT); > > } > + else if (TOKEN ("port")) > + { > + CHECK_ARGS (1) if ((rc = getNum (&service->port, toks.argv[2]))) > + return rc; > + OUTPUT_NUM (eservice, port, 2, LVS_SAW_AS_PORT); > + > + } > else > return LVS_ERROR_BADCOMMAND; > > diff -uNr piranha-0.8.2-orig/lvsd.c piranha/lvsd.c > --- piranha-0.8.2-orig/lvsd.c 2006-03-03 12:50:49.000000000 -0600 > +++ piranha/lvsd.c 2006-10-10 08:18:01.000000000 -0500 > @@ -131,7 +131,7 @@ > { > struct in_addr vip; > struct in_addr address; > - int port, protocol; > + int port, rport, protocol; > pid_t clientMonitor; /* -1 if died */ > }; > > @@ -303,7 +303,7 @@ > char *virtAddress; > char *argv[42]; > char **arg = argv; > - char portNum[20], timeoutNum[20], weightNum[20], reentryNum[20]; > + char portNum[20], rportNum[20], timeoutNum[20], weightNum[20], reentryNum[20]; > char fwmNum[20]; > pid_t child; > > @@ -313,7 +313,8 @@ > clientInfo.vip = vserver->virtualAddress; > clientInfo.address = service->address; > clientInfo.protocol = vserver->protocol; > - clientInfo.port = service->port; > + clientInfo.port = vserver->port; > + clientInfo.rport = service->port; > > rc = findClientInList (clients, *numClients, &clientInfo); > > @@ -323,7 +324,8 @@ > virtAddress = strdup (inet_ntoa (vserver->virtualAddress)); > realAddress = strdup (inet_ntoa (service->address)); > > - sprintf (portNum, "%d", service->port); > + sprintf (portNum, "%d", vserver->port); > + sprintf (rportNum, "%d", service->port); > sprintf (timeoutNum, "%d", vserver->timeout); > sprintf (weightNum, "%d", service->weight); > sprintf (reentryNum, "%d", vserver->reentryTime); > @@ -334,6 +336,8 @@ > *arg++ = realAddress; > *arg++ = (char *) "-p"; > *arg++ = portNum; > + *arg++ = (char *) "-r"; > + *arg++ = rportNum; > > if (vserver->fwmark) > { > diff -uNr piranha-0.8.2-orig/nanny.8 piranha/nanny.8 > --- piranha-0.8.2-orig/nanny.8 2006-01-11 14:35:17.000000000 -0600 > +++ piranha/nanny.8 2006-10-10 08:18:01.000000000 -0500 > @@ -14,7 +14,7 @@ > .BI "[-M|--method=" ARG "] [-U|--loadcmd=" ARG "]" > .BI "[-n|--nodaemon] [-c|--nofork] [--test-start] [--norun]" > .BI "[-p|--port=" ARG "] [-a|--reentrytime=" ARG "] [-h|--server=" ARG "]" > -.BI "[-f|--fwmark=" ARG "]" > +.BI "[-f|--fwmark=" ARG "] [-r|--rport=" ARG "]" > .BI "[-v|--verbose] [-V|--virtaddress=" ARG "] [-w|--weight=" ARG "]" > .BI "[-S|--scale=" ARG "] [-s|--send_string=" ARG "]" > .BI "[-e|--send_program=" ARG "[-x|--expect_string=" ARG "]" > @@ -27,7 +27,7 @@ > .nf > .BI "nanny [-t|--interval=" ARG "] [-U|--loadcmd=" ARG "]" > .BI "[-n|--nodaemon] [-c|--nofork] [--test-start] [--norun]" > -.BI "[-p|--port=" ARG "] [-h|--server=" ARG "] [-v|--verbose]" > +.BI "[-p|--port=" ARG "] [-h|--server=" ARG "] [-r|--rport=" ARG "] [-v|--verbose]" > .BI "[-V|--virtaddress=" ARG "] [-s|--send_string=" ARG "]" > .BI "[-x|--expect_string=" ARG "] [-R|--startup=" ARG "]" > .BI "[-D|--shutdown=" ARG "] [-u|--UDP] [-T|--suppress] [-?|--help]" > @@ -82,7 +82,8 @@ > Same as "--test-start". > .TP > .BI "-p|--port= "ARG > -Port for nanny to monitor on real server to check for service availability. > +Port used on the virtual IP address of the service to be monitored (fos), or > +the port used on the address of the virtual service to be monitored (lvsd). > If omitted, port 80 (http) is assumed. > .TP > .BI "-a|--reentrytime= "ARG > @@ -108,6 +109,10 @@ > (lvsd). If this option is set, the \fI--virtaddress\fP option will be > ignored. > .TP > +.BI "-r|--rport= "ARG > +Port for nanny to monitor on the real server to check for service > +availability. If omitted, port 80 (http) is assumed. > +.TP > .BI "-V|--virtaddress= "ARG > This is the virtual IP address of the service to be monitored (fos), or > the address of the virtual service to be monitored (lvsd). > @@ -127,7 +132,10 @@ > file. > The special token %h is replaced with the IP of each of the defined > real servers and can be used more than once though it will carry the same > -IP of the defined real server that has been selected. > +IP of the defined real server that has been selected. The special > +token %p is replaced with the port of each of the defined real servers > +and can be used more than once though it too will carry the same > +port value of the defined real server that has been selected. > eg. 'ping -c1 %h' > .TP > .BI "-s|--send_str= "ARG > diff -uNr piranha-0.8.2-orig/nanny.c piranha/nanny.c > --- piranha-0.8.2-orig/nanny.c 2006-01-12 15:54:59.000000000 -0600 > +++ piranha/nanny.c 2006-10-10 08:19:06.000000000 -0500 > @@ -652,12 +652,13 @@ > > int > shutdownDev (int flags, char *ipvsadm, char *virtualAddress, int port, > - struct in_addr *remoteAddr, int service_type) > + struct in_addr *remoteAddr, int rport, int service_type) > { > > char *argv[40]; > char **argp = argv; > char virtName[80]; > + char remoteName[80]; > char fwmStr[20]; > int rc = 0; > > @@ -681,7 +682,8 @@ > } > > *argp++ = (char *) "-r"; > - *argp++ = inet_ntoa (*remoteAddr); > + sprintf (remoteName, "%s:%d", inet_ntoa (*remoteAddr), rport); > + *argp++ = remoteName; > *argp = NULL; > > rc = runCommand (ipvsadm, flags, argv, -1); > @@ -693,11 +695,12 @@ > > int > adjustDevice (int flags, char *ipvsadm, char *virtualAddress, int port, > - struct in_addr *remoteAddr, char *routingMethod, int weight) > + struct in_addr *remoteAddr, int rport, char *routingMethod, int weight) > { > char *argv[40]; > char **argp = argv; > char virtName[80]; > + char remoteName[80]; > char weightStr[20]; > char fwmStr[20]; > > @@ -720,7 +723,8 @@ > } > > *argp++ = (char *) "-r"; > - *argp++ = inet_ntoa (*remoteAddr); > + sprintf (remoteName, "%s:%d", inet_ntoa (*remoteAddr), rport); > + *argp++ = remoteName; > *argp++ = routingMethod; > *argp++ = (char *) "-w"; > *argp++ = weightStr; > @@ -731,13 +735,14 @@ > > int > bringUpDev (int flags, char *ipvsadm, char *virtualAddress, int port, > - struct in_addr *remoteAddr, char *routingMethod, int weight, > + struct in_addr *remoteAddr, int rport, char *routingMethod, int weight, > int service_type) > { > > char *argv[40]; > char **argp = argv; > char virtName[80]; > + char remoteName[80]; > char weightStr[20]; > char fwmStr[20]; > int rc = 0; > @@ -764,7 +769,8 @@ > } > > *argp++ = (char *) "-r"; > - *argp++ = inet_ntoa (*remoteAddr); > + sprintf (remoteName, "%s:%d", inet_ntoa (*remoteAddr), rport); > + *argp++ = remoteName; > *argp++ = routingMethod; > *argp++ = (char *) "-w"; > *argp++ = weightStr; > @@ -860,10 +866,11 @@ > > int > external_check (int flags, char *send_program, char *expect_str, > - struct in_addr *remoteAddr, int timeout) > + struct in_addr *remoteAddr, int rport, int timeout) > { > char *result; > char *token = NULL; > + char tokenTemp[20]; > char temp[255]; > char *argv[40]; /* need this to build up argument list */ > char **argp = argv; > @@ -883,6 +890,10 @@ > i++; > if (strcmp (token, "%h") == 0) > token = inet_ntoa (*remoteAddr); > + else if (strcmp (token, "%p") == 0) { > + sprintf (tokenTemp, "%d", rport); > + token = tokenTemp; > + } > *argp++ = token; > token = strtok (NULL, " "); > } > @@ -897,8 +908,8 @@ > > if (flags & NANNY_FLAG_VERBOSE) > piranha_log (flags, (char *) > - "Got result (%s) from command sent to (%s)\n", > - result, inet_ntoa (*remoteAddr)); > + "Got result (%s) from command sent to (%s:%d)\n", > + result, inet_ntoa (*remoteAddr), rport); > > if ((result == NULL) && (expect_str == NULL)) { > piranha_log (flags, (char *) > @@ -908,16 +919,16 @@ > > if (result == NULL) { > piranha_log (flags, (char *) > - "Ran the external sending program to (%s) but didn't get anything back\n", > - inet_ntoa (*remoteAddr)); > + "Ran the external sending program to (%s:%d) but didn't get anything back\n", > + inet_ntoa (*remoteAddr), rport); > return 0; > } > > if (expect_str != NULL) { > if (strcmp (expect_str, result) != 0) { > piranha_log (flags, (char *) > - "Trouble. Recieved results are not what we expected from (%s)\n", > - inet_ntoa (*remoteAddr)); > + "Trouble. Recieved results are not what we expected from (%s:%d)\n", > + inet_ntoa (*remoteAddr), rport); > res = 1; > } else { > res = 0; > @@ -1005,7 +1016,7 @@ > > int > adjustWeight (int flags, char *ipvsadm, char *virtualAddress, int port, > - struct in_addr *remoteAddr, char *routingMethod, int weight, > + struct in_addr *remoteAddr, int rport, char *routingMethod, int weight, > int interval, int scale, char *loadCommand, int lastWeight) > { > int newWeight; > @@ -1020,10 +1031,10 @@ > piranha_log (flags, (char *) > "Adjusting weight of %s:%d to %d", > inet_ntoa (*remoteAddr), > - port, newWeight); > + rport, newWeight); > > if (adjustDevice (flags, ipvsadm, virtualAddress, > - port, remoteAddr, routingMethod, newWeight)) > + port, remoteAddr, rport, routingMethod, newWeight)) > return -1; > } else { > /* Querying the load information and adjusting the > @@ -1065,11 +1076,11 @@ > (char *) > "Adjusting weight of %s:%d to %d", > inet_ntoa(*remoteAddr), > - port, newWeight); > + rport, newWeight); > > if (adjustDevice (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, routingMethod, > + remoteAddr, rport, routingMethod, > newWeight)) > return -1; > > @@ -1085,7 +1096,7 @@ > > int > checkState (int flags, char *ipvsadm, char *virtualAddress, int port, > - struct in_addr *remoteAddr, char *routingMethod, int weight, > + struct in_addr *remoteAddr, int rport, char *routingMethod, int weight, > int *isActive, sigset_t *sigs, int service_type) > { > if (!nextState) > @@ -1103,7 +1114,7 @@ > /* Virtual Server */ > if (bringUpDev (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, > + remoteAddr, rport, > routingMethod, > weight, service_type)) > return -1; > @@ -1120,7 +1131,7 @@ > /* Virtual Server */ > if (shutdownDev (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, service_type)) > + remoteAddr, rport, service_type)) > return -1; > } > *isActive = 0; > @@ -1145,7 +1156,7 @@ > > int > run (int flags, char *ipvsadm, char *virtualAddress, int port, > - struct in_addr *remoteAddr, char *routingMethod, int weight, > + struct in_addr *remoteAddr, int rport, char *routingMethod, int weight, > int interval, int countThresh, int scale, char *loadCommand, > char *send_program, char *send_str, char *expect_str, > int quiesce_srv, int service_type) > @@ -1214,7 +1225,7 @@ > > if (isActive && service_type == SERV_LVS) { > if (shutdownDev (flags, ipvsadm, virtualAddress, port, > - remoteAddr, service_type)) > + remoteAddr, rport, service_type)) > exit (1); > } > > @@ -1256,10 +1267,10 @@ > if (send_program != NULL) { > isAvail = > !external_check (flags, send_program, expect_str, > - remoteAddr, interval); > + remoteAddr, rport, interval); > } else { > isAvail = > - !pinger (flags, pingSocket, remoteAddr, port, > + !pinger (flags, pingSocket, remoteAddr, rport, > interval, send_str, expect_str); > } > > @@ -1311,7 +1322,7 @@ > > piranha_log (flags, > (char *) "making %s:%d available", > - inet_ntoa (*remoteAddr), port); > + inet_ntoa (*remoteAddr), rport); > > /* compute the initial weight */ > newWeight = weight >> 5; > @@ -1322,14 +1333,14 @@ > if (adjustDevice > (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, routingMethod, > + remoteAddr, rport, routingMethod, > newWeight)) > return -1; > } else { > if (bringUpDev > (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, routingMethod, > + remoteAddr, rport, routingMethod, > newWeight, service_type)) > return -1; > isSrvUp = 1; > @@ -1350,7 +1361,7 @@ > /* Failover Services */ > piranha_log (flags, (char *) > "Exiting due to connection failure of %s:%d", > - inet_ntoa (*remoteAddr), port); > + inet_ntoa (*remoteAddr), rport); > > /* > ** Start service locally > @@ -1364,19 +1375,19 @@ > /* Virtual Services */ > piranha_log (flags, (char *) > "shutting down %s:%d due to connection failure", > - inet_ntoa (*remoteAddr), port); > + inet_ntoa (*remoteAddr), rport); > > if (quiesce_srv && isSrvUp) { > if (adjustDevice > (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, routingMethod, 0)) > + remoteAddr, rport, routingMethod, 0)) > return -1; > } else { > if (shutdownDev > (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, service_type)) > + remoteAddr, rport, service_type)) > return -1; > isSrvUp = 0; > } > @@ -1405,7 +1416,7 @@ > (interval > 20 || currCount % (20 / interval) == 0)) { > rc = adjustWeight (flags, ipvsadm, > virtualAddress, port, > - remoteAddr, routingMethod, > + remoteAddr, rport, routingMethod, > weight, interval, scale, > loadCommand, lastWeight); > if (rc < 0) > @@ -1428,7 +1439,7 @@ > alarm (0); > > rc = checkState(flags, ipvsadm, virtualAddress, > - port, remoteAddr, routingMethod, > + port, remoteAddr, rport, routingMethod, > weight, &isActive, &sigs, service_type); > if (rc < 0) > return rc; > @@ -1526,6 +1537,7 @@ > int teststart = 0; > int interval = 10; > int port = 80; > + int rport = 80; > int weight = 1; > int reentryTime = 0; > int verbose = 0; > @@ -1550,7 +1562,7 @@ > char *send_str = NULL; > char *expect_str = NULL; > > - /* Letters taken: DIMRSTUV acnpstuvwx */ > + /* Letters taken: DIMRSTUV acnprstuvwx */ > > struct poptOption options[] = { > {"interval", 't', POPT_ARG_INT, &interval, 0, > @@ -1573,6 +1585,9 @@ > N_("don't fork, but do disassociate")}, > > {"port", 'p', POPT_ARG_INT, &port, 0, > + N_("port on virtual service (default 80)")}, > + > + {"rport", 'r', POPT_ARG_INT, &rport, 0, > N_("port to check for availability (default 80)")}, > > {"reentrytime", 'a', POPT_ARG_INT, &reentryTime, 0, > @@ -1760,10 +1775,10 @@ > virtualAddress, port); > else /* service_type == SERV_LVS */ > piranha_log (flags, (char *) > - "starting LVS client monitor for %s:%d", > - virtualAddress, port); > + "starting LVS client monitor for %s:%d -> %s:%d", > + virtualAddress, port, server, rport); > > - return run (flags, vsadmPath, virtualAddress, port, &remoteAddr, > + return run (flags, vsadmPath, virtualAddress, port, &remoteAddr, rport, > routingMethod, weight, interval, countThresh, scale, > loadCommand, send_program, send_str, expect_str, > quiesce_srv, service_type); > > ------------------------------------------------------------------------ > > _______________________________________________ > Piranha-list mailing list > Piranha-list at redhat.com > https://www.redhat.com/mailman/listinfo/piranha-list -- ...................................................................... __ / / Jordi Prats C E / S / C A Dept. de Sistemes /_/ Centre de Supercomputaci? de Catalunya Gran Capit?, 2-4 (Edifici Nexus) ? 08034 Barcelona T. 93 205 6464 ? F. 93 205 6979 ? jprats at cesca.es ...................................................................... From lhh at redhat.com Wed Oct 11 20:00:32 2006 From: lhh at redhat.com (Lon Hohberger) Date: Wed, 11 Oct 2006 16:00:32 -0400 Subject: [Linux-cluster] LVS port translation In-Reply-To: <452D03E1.4060904@cesca.es> References: <452A5CD3.60500@cesca.es> <452A6842.5020608@dorm.org> <452AA973.3010904@cesca.es> <452BA3B8.6090009@dorm.org> <452D03E1.4060904@cesca.es> Message-ID: <1160596867.11134.208.camel@rei.boston.devel.redhat.com> On Wed, 2006-10-11 at 16:46 +0200, Jordi Prats wrote: > Thank you very much Brenton! > > I'll add this to the PHP interface, this way this patch will be > completly integrated to piranha. > > A general question to piranha-list: How can this patch be submited ? File a bugzilla with it :) -- Lon