天涯LVS部署

项目地址 https://code.google.com/p/tianyalvs/

添加LVS互为主备修改版：http://pan.baidu.com/s/1dEbHI0L

php的版本要求5.2

软件下载：

wget http://sourceforge.net/projects/mcrypt/files/Libmcrypt/2.5.8/libmcrypt-2.5.8.tar.gz

wget http://sourceforge.net/projects/mcrypt/files/MCrypt/2.6.8/mcrypt-2.6.8.tar.gz

wget http://sourceforge.net/projects/mhash/files/mhash/0.9.9.9/mhash-0.9.9.9.tar.gz

架构：

-- LVS web(func minion、certmaster master、rsync slave)

软件安装：LAMP、func、certmaster、rsync、keepavlied（主要是使用genhash命令）、mysql-python

    192.168.1.211

    --hostname：func-master

--LVS (func slave、certmaster minion、rsync master)

软件安装：lvs、keepavlied、func、certmaster、rsync

    192.168.1.13

    --hostname：lvs1

    192.168.1.14

    --hostname：lvs2

--realserer（func slave）

软件安装：func、certmaster、rsync

    192.168.1.200

    --hostname：lvs1

本次环境：CentOS 6.6 x64

一、LVS web

1、LAMP安装

yum -y install mysql mysql-server mysql-devel

1.1 apache:

useradd -M -s /sbin/nologin apache

./configure --prefix=/usr/local/apache --with-mysql=/usr --enable-cgi --enable-ssl --enable-so --enable-rewrite --enable-modules=all --enable-mods-shared=all --enable-cache --enable-disk-cache --enable-mem-cache --enable-deflate=shared

make; make install

1.2 安装php

安装依赖：

cd /usr/local/src

tar -zxvf libmcrypt-2.5.8.tar.gz

cd libmcrypt-2.5.8

./configure; make; make install

tar -zxvf mhash-0.9.9.9.tar.gz; cd mhash-0.9.9.9; ./configure; make; make install

tar zxvf mcrypt-2.6.8.tar.gz; cd mcrypt-2.6.8; export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH; ./configure; make; make install

mkdir /usr/lib/mysql;  ln -s /usr/lib64/mysql/libmysqlclient_r.so /usr/lib/mysql/ 

./configure --prefix=/usr/local/php --with-config-file-path=/usr/local/php/etc --with-config-file-scan-dir=/usr/local/php/etc/php.d --with-mysql=/usr --with-mysqli=/usr/bin/mysql_config-64 --with-apxs=/usr/local/apache2/bin/apxs --enable-mbstring --with-free

make; make install

php.ini：

short_open_tag = On

date.timezone = "aisa/shanghai"

2、安装certmaster、func依赖

yum -y install gcc gcc-g++ openssl openssl-devel pytho python-devel rrdtool rsync bc  keepalived

安装MySQL-python

wget https://pypi.python.org/packages/source/p/pip/pip-7.1.2.tar.gz

wget https://pypi.python.org/packages/source/s/setuptools/setuptools-19.2.tar.gz

tar -zxvf setuptools-19.2.tar.gz; cd setuptools-19.2; python setup.py install; cd ../

tar -zxvf pip-7.1.2.tar.gz; cd pip-7.1.2; python setup.py install;

pip install MySQL-python

3、安装certmaster、func

tar -zxvf pyOpenSSL-0.9.tar.gz

cd pyOpenSSL-0.9

/usr/local/bin/python setup.py install

tar -zxvf certmaster-0.25.tar.gz

cd certmaster-0.25

/usr/local/bin/python setup.py install

tar -zxvf func-0.25.tar.gz

cd func-0.25

/usr/local/bin/python setup.py install

certmaster配置：

vi /etc/certmaster/certmaster.conf

[main]

autosign = no

listen_addr =

#证书交换通讯端口

listen_port = 1998

cadir = /etc/pki/certmaster/ca

cert_dir = /etc/pki/certmaster

certroot = /var/lib/certmaster/certmaster/certs

csrroot = /var/lib/certmaster/certmaster/csrs

cert_extension = cert

sync_certs = True

func配置：

vi /etc/func/minion.conf

[main]

log_level = DEBUG

acl_dir = /etc/func/minion-acl.d

listen_addr =

#(Func通讯端口

listen_port = 1999

minion_name =

防火墙开放1998端口

启动服务

service certmaster start

chown -R apache:apache /etc/pki/certmaster/ca  apache用户需要可读

4、等LVS的certmaster、func环境安装好后，可使用下面常用操作：

certmaster-ca --list //可以查看未签名的计算机名。

certmaster-ca --sign lvs1 //对slave服务器进行签名(证书交换)。

certmaster-ca --sign `certmaster-ca --list ` //如证书请求的服务器比较多，可以这样一下子搞定。

certmaster-ca -c lvs1 //删除该主机证书

func "*" call --forks="5" command run "date" //启用5个进程来执行date命令。

func '*' ping

5、rsync + ssh 从服务器配置

cd /home/ && mkdir rsync && cd rsync

ssh-keygen -t dsa -b 1024 -f rsync-key

//生成了：rsync-key rsync-key.pub两个文件，其中rsync-key为私钥,rnync-key.pub是公钥，要传到主服务器做认证的

6、部署天涯LVS管理平台管理端

tar -zxvf TianyaLVS.tar.gz

mkdir /www

mv TianyaLVS /www

chown -R apache:apache /www/TianyaLVS/LVS /www/TianyaLVS/images/LVS /www/TianyaLVS/TianyaLVSsystem/logs    //运行apache的用户对这三个目录要有写入的权限

6.2 mysql创建数据库并导入数据

CREATE DATABASE IF NOT EXISTS tianyalvs DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

grant all privileges on tianyalvs.* to tylvs@'localhost' identified by 'tylvs_passwd';

flush privileges;

use tianyalvs;

source tianyalvs_v2.sql;

6.3 apache配置vhost

httpd.conf配置中，去掉 Include conf/extra/httpd-vhosts.conf 的注释，并修改运行apache的用户和组

apache添加vhost，这里以lvsman.lvs.com为例，要求ssl加密，访问http://lvsman.lvs.com自动跳转到https://lvsman.lvs.com

vi /usr/local/apache/conf/extra/httpd-vhosts.conf

NameVirtualHost *:80

<VirtualHost *:80>

        ServerAdmin [email protected]

        DocumentRoot "/www/TianyaLVS/"

        ServerName lvsman.lvs.com

        ServerAlias lvsman.lvs.com

        ErrorLog "logs/lvsman.lvs.com-error_log"

        CustomLog "logs/lvsman.lvs.com-access_log" common

        RewriteEngine        on

        #RewriteCond          %{HTTPS} !=on  

        RewriteRule          ^(.*)  https://%{SERVER_NAME}$1 [L,R]

        <Directory "/www/TianyaLVS">

                AllowOverride All

                allow from all

        </Directory>

</VirtualHost>

apache ssl配置请参照http://tianshili.blog.51cto.com/5050423/1735515

httpd-ssl.conf配置：

Listen 443

AddType application/x-x509-ca-cert .crt

AddType application/x-pkcs7-crl    .crl

SSLCipherSuite HIGH:MEDIUM:!MD5:!RC4

SSLProxyCipherSuite HIGH:MEDIUM:!MD5:!RC4

SSLHonorCipherOrder on 

SSLProtocol all -SSLv2 -SSLv3

SSLProxyProtocol all -SSLv2 -SSLv3

SSLPassPhraseDialog  builtin

SSLSessionCache        "shmcb:/usr/local/apache/logs/ssl_scache(512000)"

SSLSessionCacheTimeout  300

SSLMutex  "file:/usr/local/apache/logs/ssl_mutex"

<VirtualHost _default_:443>

DocumentRoot "/www/TianyaLVS"

ServerName lvsman.lvs.com:443

ServerAdmin [email protected]

ErrorLog "/usr/local/apache/logs/error_log"

TransferLog "/usr/local/apache/logs/lvsman.lvs.com.access_log"

        <Directory "/www/TianyaLVS">

                AllowOverride All

                #allow from all

                #Order                deny,allow

                #Deny                 from all

                #Allow                192.168.1.0/24

                Allow                192.168.1.20

                Allow                192.168.1.21

                # HTTP基本认证

                AuthType             basic

                AuthName             "Protected Intranet Area"

                AuthBasicProvider    file

                AuthUserFile         conf/auth_passwd

                Require              valid-user

        </Directory>

SSLEngine on

SSLCertificateFile "/usr/local/apache2/conf/server.crt"

SSLCertificateKeyFile "/usr/local/apache2/conf/server.key"

<FilesMatch "\.(cgi|shtml|phtml|php)$">

    SSLOptions +StdEnvVars

</FilesMatch>

<Directory "/usr/local/apache2/cgi-bin">

    SSLOptions +StdEnvVars

</Directory>

BrowserMatch "MSIE [2-5]" \

         nokeepalive ssl-unclean-shutdown \

         downgrade-1.0 force-response-1.0

CustomLog "/usr/local/apache2/logs/ssl_request_log" \

          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</VirtualHost>

修改项

cd TianyaLVS/TianyaLVSsystem/application/config

vi config.php

#######################

//访问URL

$config['base_url']  = "http://localhost/";

//站点目录位置

$config['webroot']  = '/www/TianyaLVS';

//管理端应用程序位置

$config['adminroot']  = '/home/LVS';

//python PATH 路径

$config['pythonroot']  = '/usr/bin/python';

//genhash bin PATH 路径

$config['genhash_bin'] = '/usr/bin/genhash';

vi database.php

#########################

$db['default']['hostname'] = "localhost";

$db['default']['username'] = "dbuser";

$db['default']['password'] = "dbpass";

$db['default']['database'] = "tianyalvs";

$db['default']['dbdriver'] = "mysql";

tar -zxvf LVS_shell.tar.gz

mv LVS /home; cd /home/LVS

#探测real server func状态

check.sh '*'

#并生成图

graph-lvs.sh

根据实际情况修改参数

引用

WORKDIR="/www/TianyaLVS/LVS/$3/performance"

RRDTOOL="/usr/bin/rrdtool"

GRAPHS="/www/TianyaLVS/images/LVS/$3"

＃获取、分析rrd

LVSgraph.py

根据实际情况修改参数，与database.php保持一致

LVSpath=/home/LVS

import MySQLdb

host='localhost'

user='dbuser'

passwd='dbpass'

db='tianyilvs'

＃func主模块

ModuleClass.py

＃同步配置LB脚本

Production_realserver.sh

根据实际情况修改参数

realserversh=/home/LVS

#LB脚本模板

rrealserver.sh.model

#同步master的rrd文件

rsync.sh

根据实际情况修改参数

KEY=/home/rsync/rsync-key

RUSER=rsyncuser

RPATH=/home/lvs-rrd/performance

LPATH=/www/TianyaLVS/LVS/$1

python /home/LVS/LVSgraph.py lvs1  //若有key要保持，则输入 yes，下同

python /home/LVS/LVSgraph.py lvs2

添加计划任务 vi /etc/crontab，去LVS服务器同步rrdtool数据过来，生效图表

#lVS Performance

*/1 * * * *     root    /usr/bin/python /home/LVS/LVSgraph.py

访问http://LVS-WEB/，完成系统安装、配置

二、LVS

lvs1：

1、安装 lvs keepalived

wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz

ln -s /usr/src/kernels/`uname -r`/ /usr/src/linux

tar zxvf ipvsadm-1.26.tar.gz

cd ipvsadm-1.26

make && make install

yum -y install keepalived

2、安装certmaster、func

yum -y install gcc gcc-g++ openssl openssl-devel pytho python-devel rrdtool rsync

tar -zxvf pyOpenSSL-0.9.tar.gz

cd pyOpenSSL-0.9

/usr/local/bin/python setup.py install

tar -zxvf certmaster-0.25.tar.gz

cd certmaster-0.25

/usr/local/bin/python setup.py install

tar -zxvf func-0.25.tar.gz

cd func-0.25

/usr/local/bin/python setup.py install

certmaster配置：

vi /etc/certmaster/certmaster.conf

[main]

autosign = no

listen_addr =

#与master端口保持一致

listen_port = 1998

cadir = /etc/pki/certmaster/ca

cert_dir = /etc/pki/certmaster

certroot = /var/lib/certmaster/certmaster/certs

csrroot = /var/lib/certmaster/certmaster/csrs

cert_extension = cert

sync_certs = False

vi /etc/certmaster/minion.conf

[main]

certmaster = func-master.server.com

#与master端口保持一致

certmaster_port = 1998

log_level = DEBUG

cert_dir = /etc/pki/certmaster

func配置：

vi /etc/func/minion.conf

[main]

log_level = DEBUG

acl_dir = /etc/func/minion-acl.d

listen_addr =

#与master端口保持一致

listen_port = 1999

#slave主机名

minion_name =lvs1

启动服务

chkconfig --level 345 certmaster on

service certmaster start

chkconfig --level 345 funcd on

service funcd start

防火墙对LVS-WEB开放1998(certmaster)、1999(func)端口

//安装完毕后一定要重启服务器操作系统，不然服务器端有时看不到该主机的证书请求。

3、rsync + ssh 主服务器配置

#useradd rsyncuser

#su rsyncuser

#cd /home/rsyncuser

#if [ ! -d .ssh ]; then mkdir .ssh ; chmod 700 .ssh ; fi

#mv rsync-key.pub(从服务器生成的公钥) .ssh/

#cd .ssh/

SSH1: #cat rsync-key.pub >> authorized_keys

SSH2: #ssh-keygen -X -f rsync-key.pub >> authorized_keys2

chmod 600 authorized_keys //这一步一定要做，特殊权限要求。

我用的是SSH2，但用SSH2的方式有一个提示长度越界的错误，那就用SSH1的方法也没问题。

4、配置 rrdtool产生数据

tar -zxvf lvs-rrd.tar.gz

mv lvs-rrd /home

确定 /home/lvs-rrd/lvs.rrd.update里的路径

RRDTOOL="/usr/bin/rrdtool"

IPVSADM="/sbin/ipvsadm"

WORKDIR="/home/lvs-rrd/performance"

最后添加计划任务

vi /etc/crontab

*/2 * * * *     root    /home/lvs-rrd/lvs.rrd.update 2> /dev/null > /dev/null

lvs2的安装配置参照lvs1