2. HTTP load-balancing with cookie prefixing and high availability
==================================================================
2. HTTP load-balancing with cookie prefixing and high availability
==================================================================
Now you don't want to add more cookies, but rather use existing ones. The
application already generates a "JSESSIONID" cookie which is enough to track
sessions, so we'll prefix this cookie with the server name when we see it.
Since the load-balancer becomes critical, it will be backed up with a second
one in VRRP mode using keepalived under Linux.
Download the latest version of keepalived from this site and install it
on each load-balancer LB1 and LB2 :
http://www.keepalived.org/
You then have a shared IP between the two load-balancers (we will still use the
original IP). It is active only on one of them at any moment. To allow the
proxy to bind to the shared IP on Linux 2.4, you must enable it in /proc :
# echo 1 >/proc/sys/net/ipv4/ip_nonlocal_bind
shared IP=192.168.1.1
192.168.1.3 192.168.1.4 192.168.1.11-192.168.1.14 192.168.1.2
-------+------------+-----------+-----+-----+-----+--------+----
| | | | | | _|_db
+--+--+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)
| LB1 | | LB2 | | A | | B | | C | | D | (___)
+-----+ +-----+ +---+ +---+ +---+ +---+ (___)
haproxy haproxy 4 cheap web servers
keepalived keepalived
Config on both proxies (LB1 and LB2) :
--------------------------------------
listen webfarm 192.168.1.1:80
mode http
balance roundrobin
cookie JSESSIONID prefix
option httpclose
option forwardfor
option httpchk HEAD /index.html HTTP/1.0
server webA 192.168.1.11:80 cookie A check
server webB 192.168.1.12:80 cookie B check
server webC 192.168.1.13:80 cookie C check
server webD 192.168.1.14:80 cookie D check
Notes: the proxy will modify EVERY cookie sent by the client and the server,
so it is important that it can access to ALL cookies in ALL requests for
each session. This implies that there is no keep-alive (HTTP/1.1), thus the
"httpclose" option. Only if you know for sure that the client(s) will never
use keep-alive (eg: Apache 1.3 in reverse-proxy mode), you can remove this
option.
Configuration for keepalived on LB1/LB2 :
-----------------------------------------
vrrp_script chk_haproxy { # Requires keepalived-1.1.13
script "killall -0 haproxy" # cheaper than pidof
interval 2 # check every 2 seconds
weight 2 # add 2 points of prio if OK
}
vrrp_instance VI_1 {
interface eth0
state MASTER
virtual_router_id 51
priority 101 # 101 on master, 100 on backup
virtual_ipaddress {
192.168.1.1
}
track_script {
chk_haproxy
}
}
Description :
-------------
- LB1 is VRRP master (keepalived), LB2 is backup. Both monitor the haproxy
process, and lower their prio if it fails, leading to a failover to the
other node.
- LB1 will receive clients requests on IP 192.168.1.1.
- both load-balancers send their checks from their native IP.
- if a request does not contain a cookie, it will be forwarded to a valid
server
- in return, if a JESSIONID cookie is seen, the server name will be prefixed
into it, followed by a delimitor ('~')
- when the client comes again with the cookie "JSESSIONID=A~xxx", LB1 will
know that it must be forwarded to server A. The server name will then be
extracted from cookie before it is sent to the server.
- if server "webA" dies, the requests will be sent to another valid server
and a cookie will be reassigned.
Flows :
-------
(client) (haproxy) (server A)
>-- GET /URI1 HTTP/1.0 ------------> |
( no cookie, haproxy forwards in load-balancing mode. )
| >-- GET /URI1 HTTP/1.0 ---------->
| X-Forwarded-For: 10.1.2.3
| <-- HTTP/1.0 200 OK -------------<
( no cookie, nothing changed )
<-- HTTP/1.0 200 OK ---------------< |
>-- GET /URI2 HTTP/1.0 ------------> |
( no cookie, haproxy forwards in lb mode, possibly to another server. )
| >-- GET /URI2 HTTP/1.0 ---------->
| X-Forwarded-For: 10.1.2.3
| <-- HTTP/1.0 200 OK -------------<
| Set-Cookie: JSESSIONID=123
( the cookie is identified, it will be prefixed with the server name )
<-- HTTP/1.0 200 OK ---------------< |
Set-Cookie: JSESSIONID=A~123 |
>-- GET /URI3 HTTP/1.0 ------------> |
Cookie: JSESSIONID=A~123 |
( the proxy sees the cookie, removes the server name and forwards
to server A which sees the same cookie as it previously sent )
| >-- GET /URI3 HTTP/1.0 ---------->
| Cookie: JSESSIONID=123
| X-Forwarded-For: 10.1.2.3
| <-- HTTP/1.0 200 OK -------------<
( no cookie, nothing changed )
<-- HTTP/1.0 200 OK ---------------< |
( ... )
Hints :
-------
Sometimes, there will be some powerful servers in the farm, and some smaller
ones. In this situation, it may be desirable to tell haproxy to respect the
difference in performance. Let's consider that WebA and WebB are two old
P3-1.2 GHz while WebC and WebD are shiny new Opteron-2.6 GHz. If your
application scales with CPU, you may assume a very rough 2.6/1.2 performance
ratio between the servers. You can inform haproxy about this using the "weight"
keyword, with values between 1 and 256. It will then spread the load the most
smoothly possible respecting those ratios :
server webA 192.168.1.11:80 cookie A weight 12 check
server webB 192.168.1.12:80 cookie B weight 12 check
server webC 192.168.1.13:80 cookie C weight 26 check
server webD 192.168.1.14:80 cookie D weight 26 check
========================================================
2.1 Variations involving external layer 4 load-balancers
========================================================
Instead of using a VRRP-based active/backup solution for the proxies,
they can also be load-balanced by a layer4 load-balancer (eg: Alteon)
which will also check that the services run fine on both proxies :
| VIP=192.168.1.1
+----+----+
| Alteon |
+----+----+
|
192.168.1.3 | 192.168.1.4 192.168.1.11-192.168.1.14 192.168.1.2
-------+-----+------+-----------+-----+-----+-----+--------+----
| | | | | | _|_db
+--+--+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)
| LB1 | | LB2 | | A | | B | | C | | D | (___)
+-----+ +-----+ +---+ +---+ +---+ +---+ (___)
haproxy haproxy 4 cheap web servers
Config on both proxies (LB1 and LB2) :
--------------------------------------
listen webfarm 0.0.0.0:80
mode http
balance roundrobin
cookie JSESSIONID prefix
option httpclose
option forwardfor
option httplog
option dontlognull
option httpchk HEAD /index.html HTTP/1.0
server webA 192.168.1.11:80 cookie A check
server webB 192.168.1.12:80 cookie B check
server webC 192.168.1.13:80 cookie C check
server webD 192.168.1.14:80 cookie D check
The "dontlognull" option is used to prevent the proxy from logging the health
checks from the Alteon. If a session exchanges no data, then it will not be
logged.
Config on the Alteon :
----------------------
/c/slb/real 11
ena
name "LB1"
rip 192.168.1.3
/c/slb/real 12
ena
name "LB2"
rip 192.168.1.4
/c/slb/group 10
name "LB1-2"
metric roundrobin
health tcp
add 11
add 12
/c/slb/virt 10
ena
vip 192.168.1.1
/c/slb/virt 10/service http
group 10
Note: the health-check on the Alteon is set to "tcp" to prevent the proxy from
forwarding the connections. It can also be set to "http", but for this the
proxy must specify a "monitor-net" with the Alteons' addresses, so that the
Alteon can really check that the proxies can talk HTTP but without forwarding
the connections to the end servers. Check next section for an example on how to
use monitor-net.
============================================================
2.2 Generic TCP relaying and external layer 4 load-balancers
============================================================
Sometimes it's useful to be able to relay generic TCP protocols (SMTP, TSE,
VNC, etc...), for example to interconnect private networks. The problem comes
when you use external load-balancers which need to send periodic health-checks
to the proxies, because these health-checks get forwarded to the end servers.
The solution is to specify a network which will be dedicated to monitoring
systems and must not lead to a forwarding connection nor to any log, using the
"monitor-net" keyword. Note: this feature expects a version of haproxy greater
than or equal to 1.1.32 or 1.2.6.
| VIP=172.16.1.1 |
+----+----+ +----+----+
| Alteon1 | | Alteon2 |
+----+----+ +----+----+
192.168.1.252 | GW=192.168.1.254 | 192.168.1.253
| |
------+---+------------+--+-----------------> TSE farm : 192.168.1.10
192.168.1.1 | | 192.168.1.2
+--+--+ +--+--+
| LB1 | | LB2 |
+-----+ +-----+
haproxy haproxy
Config on both proxies (LB1 and LB2) :
--------------------------------------
listen tse-proxy
bind :3389,:1494,:5900 # TSE, ICA and VNC at once.
mode tcp
balance roundrobin
server tse-farm 192.168.1.10
monitor-net 192.168.1.252/31
The "monitor-net" option instructs the proxies that any connection coming from
192.168.1.252 or 192.168.1.253 will not be logged nor forwarded and will be
closed immediately. The Alteon load-balancers will then see the proxies alive
without perturbating the service.
Config on the Alteon :
----------------------
/c/l3/if 1
ena
addr 192.168.1.252
mask 255.255.255.0
/c/slb/real 11
ena
name "LB1"
rip 192.168.1.1
/c/slb/real 12
ena
name "LB2"
rip 192.168.1.2
/c/slb/group 10
name "LB1-2"
metric roundrobin
health tcp
add 11
add 12
/c/slb/virt 10
ena
vip 172.16.1.1
/c/slb/virt 10/service 1494
group 10
/c/slb/virt 10/service 3389
group 10
/c/slb/virt 10/service 5900
group 10
Special handling of SSL :
-------------------------
Sometimes, you want to send health-checks to remote systems, even in TCP mode,
in order to be able to failover to a backup server in case the first one is
dead. Of course, you can simply enable TCP health-checks, but it sometimes
happens that intermediate firewalls between the proxies and the remote servers
acknowledge the TCP connection themselves, showing an always-up server. Since
this is generally encountered on long-distance communications, which often
involve SSL, an SSL health-check has been implemented to workaround this issue.
It sends SSL Hello messages to the remote server, which in turns replies with
SSL Hello messages. Setting it up is very easy :
listen tcp-syslog-proxy
bind :1514 # listen to TCP syslog traffic on this port (SSL)
mode tcp
balance roundrobin
option ssl-hello-chk
server syslog-prod-site 192.168.1.10 check
server syslog-back-site 192.168.2.10 check backup
=========================================================