使用haproxy来实现sphinx负载均衡与健康监测

使用haproxy来实现sphinx负载均衡与健康监测  引自http://www.ttlsa.com/archives/295#more-295
环境如下：
【node1】
Sphinx:192.168.1.189
【node2】
Sphinx:192.168.1.101
【node3】
Haproxy:192.168.1.103

【node3】上配置：
# apt-get install make gcc g++ automake libtool

1.安装
# wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.18.tar.gz
# tar zxvf haproxy-1.4.18.tar.gz -C ../software/
# cd haproxy-1.4.18
# make TARGET=linux26 PREFIX=/usr/local/haproxy-1.4.18
# make install PREFIX=/usr/local/haproxy-1.4.18

配置
# cd /usr/local/haproxy-1.4.18
# vim haproxy.cfg
global
        log 127.0.0.1   local0 info  //日志[err warning info debug]
        maxconn 4096  //单个进程的最大连接数
        pidfile /usr/local/haproxy-1.4.18/log/haproxy.pid
        user haproxy
        group haproxy
        daemon //守护进程运行
        nbproc 4 //作为守护进程运行时，创建多少个进程
defaults
        log     global
        mode    http
        option  dontlognull //不记录空连接
        option  log-health-checks //记录健康检测日志
        retries 3 //连接失败后的重连次数
        option  redispatch //在连接失败或断开情况下，允许当前会话被重新分发
        maxconn 2000 //最大连接数
        timeout connect 5000ms 
        timeout client  30000ms
        timeout server  30000ms
listen  haproxy-status 0.0.0.0:8888
        mode          http
        option        httplog
        stats uri     /hasphinx  //统计页面url
        stats realm   Global\ statistics
        stats auth    hasphinx:hasphinx  //登录用户和密码
listen  proxy-sphinx-1 0.0.0.0:3312
        mode   tcp
        balance roundrobin  //服务器分配算法
        option tcpka
        option httpchk
        server node1 192.168.1.189:3312 weight 1 check port 9312 inter 1s rise 2 fall 2
        server node2 192.168.1.101:3312 weight 1 check port 9312 inter 1s rise 2 fall 2
 //后台服务器
 #weight 服务器权重
 #check port 检测端口
 #inter 设置连续的两次健康检测间的时间
 #rise 指定多少次连接成功的健康检测后，可认定该服务器可用
 #fall 指定多少次失败的健康检测后，可认定该服务器当掉

配置日志
# vim /etc/rsyslog.d/haproxy.conf
$ModLoad imudp
$UDPServerRun 514
local0.*        /var/log/haproxy.log
重启日志服务
# restart rsyslog

【node1/node2】上配置：
1.健康检测
# apt-get install xinetd
# vim /etc/inetd.conf
sphinxchk       stream  tcp     wait    root    /usr/local/scripts/sphinxchk.sh
注释：
a.服务名称：inetd通过查询/etc/services获得该服务相关信息
b.套接字类型：该服务使用的通信协议tcp用stream，udp用dgram
c.Inetd是否等待守护进程结束才继续接管端口。Wait表示等待，nowait表示不等待，inetd每次接到一个请求就启动守护进程的新副本
d.运行该守护进程的用户
e.守护进程二进制文件的路径以及参数

# vim /etc/xinetd.d/sphinxchk
service sphinxchk
{
        flags           = REUSE
        socket_type     = stream
        port            = 9312
        wait            = no
        user            = root
        server         = /usr/local/scripts/sphinxchk.sh
        log_on_failure  += USERID
        disable         = no
        only_from       = 192.168.1.0/24
}

sphinx配置省略

测试：
将sphinx源码api目录下的sphinxapi.py test.py拷贝到测试服务器上。
# vim sphinx-test.sh
#!/bin/bash
i=0
while [ "$i" -lt "99999" ]
do
  /usr/bin/python  /root/test.py -a $*
  let i=$i+1
done
# ./sphinx-test.sh php aaa bbb ccc dddd
Kill掉node1上的sphinx服务
Oct 13 01:59:59 localhost haproxy[12117] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 32ms, status: 1/2 UP.
Oct 13 01:59:59 localhost haproxy[12119] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 31ms, status: 1/2 UP.
Oct 13 01:59:59 localhost haproxy[12118] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 31ms, status: 1/2 UP.
Oct 13 01:59:59 localhost haproxy[12120] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 31ms, status: 1/2 UP.
Oct 13 02:00:00 localhost haproxy[12117] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 35ms, status: 0/2 DOWN.
Oct 13 02:00:00 localhost haproxy[12117] Server proxy-sphinx-1/node1 is DOWN. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Oct 13 02:00:00 localhost haproxy[12119] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 32ms, status: 0/2 DOWN.
Oct 13 02:00:00 localhost haproxy[12119] Server proxy-sphinx-1/node1 is DOWN. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Oct 13 02:00:00 localhost haproxy[12118] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 30ms, status: 0/2 DOWN.
Oct 13 02:00:00 localhost haproxy[12118] Server proxy-sphinx-1/node1 is DOWN. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Oct 13 02:00:00 localhost haproxy[12120] Health check for server proxy-sphinx-1/node1 failed, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 32ms, status: 0/2 DOWN.
Oct 13 02:00:00 localhost haproxy[12120] Server proxy-sphinx-1/node1 is DOWN. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
请求将会发送到node2上面。（将sphinx的查询日志打开进行观察）