Firewall marks not working as expected?

[Brenton Rothchild]

Hi,

I'm trying to get piranha (0.8.1) in a LVS-NAT setup to
use firewall marks to bundle HTTP and HTTPS traffic
together with persistence.

However, when pulse starts up, I'm getting messages
like this (when using "/usr/sbin/pulse -c /etc/sysconfig/ha/lvs.cf -n -v"):

lvs: starting virtual service server_119_129_https active: 443
Service already exists
lvs: ipvsadm failed for virtual server server_119_129_https!

The ipvsadm command seems to work for the first entry (port 80,
see lvs.cf below), and nanny processes are started for each of
the 3 real servers for the HTTP process, but the second entry
(for port 443) isn't - due to ipvsadm failing as shown in the
above log messages.

Looking at the code, it looks like when using firewall marks,
the ipvsadm command isn't getting a full "ip_address:port" definition, as
in:

<From piranha-0.8.1/lvsd.c, in startVirtualServer()>

   if (vserver->fwmark)
     {
       *arg++ = (char *) "-f";
       (void) sprintf (fwmNum, "%d", vserver->fwmark);
       *arg++ = fwmNum;
     }
   else
     {
       switch (vserver->protocol)
         {
         case IPPROTO_UDP:
           *arg++ = (char *) "-u";
           break;

         case IPPROTO_TCP:
         default:
           *arg++ = (char *) "-t";
           break;
         }
       sprintf (virtAddress, "%s:%d", inet_ntoa (vserver->virtualAddress),
                vserver->port);
       *arg++ = virtAddress;
     }


I'm trying to use the examples given in the RHCS manual, as per
http://www.redhat.com/docs/manuals/csgfs/browse/rh-cs-en/s1-lvs-multi.html:

/sbin/iptables -t mangle -A PREROUTING -p tcp \
	    -d 10.0.119.129/32 --dport 80 -j MARK --set-mark 80
/sbin/iptables -t mangle -A PREROUTING -p tcp \
             -d 10.0.119.129/32 --dport 443 -j MARK --set-mark 80

After trying the ipvsadm commands from the command line myself, the problem
appears to be since both services are defined only by "-f 80", ipvsadm
is assuming they're the same thing and issuing the "Service already exists"
message, as in:

# Using fwmarks
/sbin/ipvsadm -A -f 80 -s wlc -p 60 -M 255.255.255.255
/sbin/ipvsadm -A -f 80 -s wlc -p 60 -M 255.255.255.255

# versus:
# Not using fwmarks
/sbin/ipvsadm -A -t 10.0.119.129:80 -s wlc -p 60 -M 255.255.255.255
/sbin/ipvsadm -A -t 10.0.119.129:443 -s wlc -p 60 -M 255.255.255.255

Here is my lvs.cf file:

service = lvs
primary = 10.0.0.1
backup = 10.0.0.2
backup_active = 1
heartbeat = 1
heartbeat_port = 1050
keepalive = 6
deadtime = 18

rsh_command = ssh

network = nat
nat_router = 192.168.15.254 eth0:1

virtual server_119_129_http {
         address = 10.0.119.129 eth1:129
         active = 1
         load_monitor = uptime
         timeout = 5
         reentry = 10
         port = 80
         send = "GET / HTTP/1.0\r\n\r\n"
         expect = "HTTP"
         scheduler = wlc
         persistent = 60
         pmask = 255.255.255.255
         fwmark = 80
         protocol = tcp

         server app-1 {
                 address = 192.168.5.1
                 active = 1
                 weight = 1
         }

         server app-2 {
                 address = 192.168.5.2
                 active = 1
                 weight = 1
         }

         server app-3 {
                 address = 192.168.5.3
                 active = 1
                 weight = 1
         }
}

virtual server_119_129_https {
         address = 10.0.119.129 eth1:129
         active = 1
         load_monitor = uptime
         timeout = 5
         reentry = 10
         port = 443
         send = "GET / HTTP/1.0\r\n\r\n"
         expect = "HTTP"
         scheduler = wlc
         persistent = 60
         pmask = 255.255.255.255
         fwmark = 80
         protocol = tcp

         server app-1 {
                 address = 192.168.5.1
                 active = 1
                 weight = 1
         }

         server app-2 {
                 address = 192.168.5.2
                 active = 1
                 weight = 1
         }

         server app-3 {
                 address = 192.168.5.3
                 active = 1
                 weight = 1
         }
}


Am I doing something completely wrong here?  Anyone
have any suggestions?


Thanks!
-Brenton Rothchild