注意：WEB服务器和数据库需要分离，同时WEB服务器也需要编译安装MySQL。
做集群架构的重要思想就是找到主干，从主干区域向外延展。
WEB服务器： apache nginx 本地做三个产品 dedecms workprocess discuz 将用户存放图片，附件的目录挂载到nfs服务器的共享目录上
NFS服务器 本地做三个共享目录，实现将用户上传的图片及附件分别存放到对应的目录上，
使用sersync与备份服务器实现实时同步，批量分发密钥及脚本，hosts文件(实际生产环境下，在同一局域网下，hosts文件通常保持一致)，
MySQL服务器：用于用户存放数据的服务器，
Backup服务器：用于备份的服务器，防止其他服务器宕机、感染病毒、等等数据丢失。同时要将每天备份的内容通过邮件发送给管理员，确保数据备份成功。
我的主干思想就是先配置LAMP和LNMP服务器，之后向外延展配置nfs服务器及MySQL服务器，然后将所有需要备份的数据打包好，配置backu备份服务器，最后做nginx负载均衡服务器，如果有精力又有能力的情况下，继续延伸一个nginx的高可用(提示ngixn高可用服务使用的是VRRP技术)
1.LAMP(192.168.190.20)
(1)tar xvf /apache-2.2.27 cd apache-2.2.27 编译安装 ./configure \ --prefix=/application/appache2.2.27 \ 安装目录 --enable-deflate \ 压缩安装 --enable-expires \ 过期 缓存时间 --enable-headers \ --enable-modules=most \ 模块激活 --enable-so \ --with-mpm=worker \ apache的两种模式:worker,prefork --enable-rewrite && make &&make install (注意编译环境下换行后边不能存在空格，上述编译添加了注释，如果粘贴请自行删除，手打忽略) ln -s /application/apache-2.2.27 /application/apache echo “a ,s blog. Hi,i'm a ,My blog address is ” > /application/apache/htdos/index.html /application/apache/bin/apachectl graceful
浏览器输入192.168.190.20 会出现
Hi,i'm a ,My blog address is 等字样说明apache服务安装成功
(2)安装数据库msyql
解压编译安装 ，编译过程略长，安装结束后进行检查做软链接
创建MySQL虚拟用户和用户组
groupadd mysql cat /etc/group useradd -g mysql -M -s /sbin/nologin mysql id mysql 编译安装MySQL ./configure \ --prefix=/application/mysql5.1.72 \ --with-unix-socket-path=/application/mysql5.1.72/tmp/mysql.sock \ --localstatedir=/application/mysql5.1.72/data \ --enable-assembler \ --enable-thread-safe-client \ --with-mysqld-user=mysql \ --with-big-tables \ --without-debug \ --with-pthread \ --enable-assembler \ --with-extra-charsets=complex \ --with-readline \ --with-ssl \ --with-embedded-server \ --enable-local-infile \ --with-plugins=partition,innobase \ --with-mysqld-ldflags=-all-static \ --with-client-ldflags=-all-static #--with-plugin-PLUGIN \ make && make install
echo $? 检查编译是否成功
ln -s /application/mysql5.1.72/ /application/mysql 创建软链接
复制配置mysql的配置文件
cd mysql-5.1.72/support-files/ ls cp -p my-small.cnf /etc/my.cnf chown -R mysql.mysql /application/mysql 授权MySQL用户管理权限 初始化mysql /application/mysql/bin/mysql_install_db --basedir=/application/mysql --datadir=/application/mysql/data/ --user=mysql ##两个OK即为初始化成功 /application/mysql/bin/mysqld_safe & 启动mysql netstat -lntup|grep mysqld ##查看MySQL服务是否启动成功 mysqladmin -u root password '123456' ##设置MySQL用户密码
(3)安装完apache和mysql之后再安装PHP(注意php配合apache是以模块的方式存在)
yum install -y openssl-devel tar -xvf php-5.3.27.tar.gz cd php-5.3.27.tar.gz ./configure \ --prefix=/application/php5.3.27 \ //注意php的安装目录 --with-apxs2=/application/apache/bin/apxs \ //注意apache的安装目录 --with-mysql=/application/mysql \ --with-xmlrpc \ --with-openssl \ --with-zlib \ --with-freetype-dir \ --with-gd \ --with-jpeg-dir \ --with-png-dir \ --with-iconv=/usr/local/libiconv \ --enable-short-tags \ --enable-sockets \ --enable-zend-multibyte \ --enable-soap \ --enable-mbstring \ --enable-static \ --enable-gd-native-ttf \ --with-curl \ --with-xsl \ --enable-ftp \ --with-libxml-dir && make && make install
ln -s /application/php5.3.27/ /application/php ##做软链接去版本号 cp /application/apache/conf/httpd.conf /application/apache/conf/httpd.conf.bak.1 vim /application/apache/conf/httpd.conf #修改主配置文件 cd /application/apache/conf diff httpd.conf httpd.conf.bak.1 67,68c67,68 < User www < Group www --- > User daemon > Group daemon 149c149 < DirectoryIndex index.php index.html --- > DirectoryIndex index.html 292,294c292 < AddType application/x-httpd-php .php .phtml < AddType application/x-httpd-php-source .phps --- > 401c401 < Include conf/extra/httpd-vhosts.conf --- ># Include conf/extra/httpd-vhosts.conf: 423424 Options -Indexes FollowSymLinks 425 AllowOverride None 426 Order allow,deny 427 Allow from all 428
创建对应的apache的虚拟用户www
useradd www -s /sbin/nologin -M
id www
cd /application/apache/conf/extra
vim httpd-vhosts.conf
NameVirtualHost *:80 # # VirtualHost example: # Almost any Apache directive may go into a VirtualHost container. # The first VirtualHost section is used for all requests that do not # match a ServerName or ServerAlias in anyblock. # ServerAdmin [email protected] DocumentRoot "/data0/www/cms" ServerName cms.etiantian.org ServerAlias etiantian.org ErrorLog "logs/dummy-host.example.com-error_log" CustomLog "|/usr/local/sbin/cronolog /app/logs/access_cms_%Y%m%d.log" combined ServerAdmin [email protected] DocumentRoot "/data0/www/bbs" ServerName bbs.etiantian.org ErrorLog "logs/dummy-host.example.com-error_log" CustomLog "|/usr/local/sbin/cronolog /app/logs/access_bbs_%Y%m%d.log" combined ServerAdmin [email protected] DocumentRoot "/data0/www/blog" ServerName blog.etiantian.org ErrorLog "logs/dummy-host.example.com-error_log" CustomLog "|/usr/local/sbin/cronolog /app/logs/access_blog_%Y%m%d.log" combined
如果有错误则查看错误日志
cd /application/appache/logs/
建立站点目录
mkdir /data0/{www,blog,bbs}
for n in www blog bbs ;do echo "$n.etiantian.org" > /data0/$n/index.html;done
/application/apache/bin/apachectl -t #检查配置文件的语法
/application/apache/bin/apachectl graceful #平滑重启apache
本地做hosts解析 访问三个网站 ，查看基于域名的虚拟主机是否配置成功
Dedecms,Discuz,workprocess三个产品解压之后自行安装(安装完MySQL数据库再进行安装)
2.LNMP(192.168.190.10)
(1)安装nginx服务
编译nginx前 需安装
1.pcre pcre-devel
yum install -y pcre pcre-devel
2.openssl
yum install openssl openssl-devel -y
编译安装nginx
./configure --user=nginx --group=nginx --prefix=/application/nginx1.6.2 --with-http_stub_status_module --with-http_ssl_module make && make install ln -s /application/nginx1.6.2/ /application/nginx
启动nginx
/application/nginx/sbin/nginx
lsof -I :80 #查看nginx服务是否启动成功
(2)安装MySQL数据库
编译安装MySQL
./configure --prefix=/application/mysql5.1.72 --with-unix-socket-path=/application/mysql5.1.72/tmp/mysql.sock --localstatedir=/application/mysql5.1.72/data --enable-assembler
--enable-thread-safe-client --with-mysqld-user=mysql --with-big-tables --without-debug --with-pthread --enable-assembler --with-extra-charsets=complex --with-readline
--with-ssl --with-embedded-server --enable-local-infile --with-plugins=partition,innobase --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static make && make install
echo $?
数据库初始化:
chown -R mysql.mysql /application/mysql 授权MySQL用户管理权限
初始化mysql
/application/mysql/bin/mysql_install_db --basedir=/application/mysql --datadir=/application/mysql/data/ --user=mysql
//初始化显示两个OK 即为初始化成功
(3)安装PHP(PHP配合nginx是以守护进程的方式存在工作的)
(安装php之前需要安装所需的包 yum install zlib libxml libjpeg freetype libpng gd curl libiconv zlib-devel libxml2-devel libjpeg-devel freetype-devel libpng-devel gd-devel curl-devel -y)
tar zxf libiconv-1.14.tar.gz cd libiconv-1.14 ./configure --prefix=/usr/local/libiconv //然后进行编译安装 make && make install
安装相关依赖
Libmcrypt 、 mhash 、mcrypt
安装php（编译之前首先安装libxslt* ,否则会报错） 解压
./configure --prefix=/application/php5.3.27 --with-mysql=/application/mysql --with-iconv-dir=/usr/local/libiconv --with-freetype-dir --with-jpeg-dir --with-png-dir --with-zlib
--with-libxml-dir=/usr --enable-xml --disable-rpath --enable-safe-mode --enable-bcmath --enable-shmop --enable-sysvsem --enable-inline-optimization --with-curl
--with-curlwrappers --enable-mbregex --enable-fpm --enable-mbstring --with-mcrypt --with-gd --enable-gd-native-ttf --with-openssl --with-mhash --enable-sockets --with-xmlrpc
--enable-zip --enable-soap --enable-short-tags --enable-zend-multibyte --enable-static --with-xsl --with-fpm-user=nginx --with-fpm-group=nginx --enable-ftp && make && make install
上传修改好的启动文件php-frm.conf(软件包组里面有修改完整的) à上传到/application/php/etc
创建日志文件
mkdir /app/logs //不创建的话检查语法的时候会报错
/application/php/sbin/php-fpm -t //检查语法
/application/php/sbin/php-fpm //启动php
在rc.local里设置开机自启动
Mysql php nginx
##优化配置文件
cat /application/nginx/conf/
cat nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
include extra/cms.conf;
include extra/bbs.conf;
include extra/blog.conf;
}
mdkir extra
touch {bbs,blog,cms}.conf
cat cms.conf
server {
listen 80;
server_name cms.etiantian.org;
root /data0/www/cms;
index index.php index.html index.htm;
location ~ \.(php|php5)?$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi.conf;
}
}
cat bbs.conf
server {
listen 800;
server_name bbs.etiantian.org;
root /data0/www/bbs;
index index.php index.html index.htm;
location ~ \.(php|php5)?$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi.conf;
}
}
cat blog,conf
server {
listen 8000;
server_name blog.etiantian.org;
root /data0/www/blog;
index index.php index.html index.htm;
location ~ \.(php|php5)?$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi.conf;
}
}
mkdir /data0/{www,blog,bbs}
for n in www blog bbs ;do echo "$n.etiantian.org" > /data0/$n/index.html;done
Discuz,dedecms,workprocess三个产品自行安装放入站点目录 bbs,www,blog
3.NFS服务器(192.168.190.30)
(1)安装nfs和rpcbind(nfs服务是靠rpcbind转发端口的)
yum install nfs-utils rpcbind -y
echo “/etc/init.d/nfs restart” >/etc/rc.local //设置nfs开机自启动使用chkconfig同样
vim /etc/exports
#shared storage for LAMP,LNMP /data0 192.168.190.10(rw,async) 192.168.190.20(rw.async)
(2)批量分发
ssh-copy-id -i 是可以实现小环境下的密钥分发但是如果上百台服务器，就需要开发脚本进行自动化分发密钥(附件里有开发好的脚本，仅供参考)
密钥分发完毕，分发本地的hosts文件(实际生产环境下统一内网下，hosts本地解析一致是很有必要的)
vim /etc/rsync.password
123456
chmod 600 /etc/rsync.password
(3)sersync实现与备份备份服务器的实时同步
安装sersync
cd /tools
uzip sersync2.5.4_64bit_binary_stable_final.tar
mv sersync2 /usr/local/sersync
cd /usr/local/sersync/conf
echo ‘export PATH=$PATH:/usr/local/sersync/bin’ >>/etc/profile
vim /usr/local/sersync/conf/www_confxml.xml (需要哪个目录rsync服务器同步就写哪个目录,这里以博客workprocess为例name代表rsync服务端的模块名称 ip就是目标IP地址)
2425 26
echo ‘sersync -r -d -o /usr/local/sersync/conf/www_confxml.xml’ >> /etc/rc.local
至此sersync与备份服务器实时同步也完成了。
4.安装MySQL服务器(192.168.190.40)，创建你所做的产品的数据库，创建每个产品所使用的数据库用户，及授权用户。
创建MySQL虚拟用户 useradd -g mysql -M -s /sbin/nologin mysql
(1)编译安装MySQL数据库
tar xvf mysql5.1.72.tar.gz
cd mysql5.1.72
./configure --prefix=/application/mysql5.1.72 --with-unix-socket-path=/application/mysql5.1.72/tmp/mysql.sock --localstatedir=/application/mysql5.1.72/data --enable-assembler
--enable-thread-safe-client --with-mysqld-user=mysql --with-big-tables --without-debug --with-pthread --enable-assembler --with-extra-charsets=complex --with-readline
--with-ssl --with-embedded-server --enable-local-infile --with-plugins=partition,innobase --with-mysqld-ldflags=-all-static --with-client-ldflags=-all-static
ln -s /application/mysql5.1.72/ /application/mysql
/application/mysql/bin/mysql_install_db --basedir=/application/mysql --datadir=/application/mysql/data/ --user=mysql ##初始化数据库
cd /tools/msyql5.1.71
\cp support-files/mysql.server /etc/init.d/mysqld
/etc/init.d/mysqld start
netstat -lntup |grep mysql #查看MySQL3306端口是否开启
mysqladmin -u root password '123456' ## 设置数据库用户名和密码
(2)进入数据库创建用户并进行授权，创建用户所用的数据库cms bbs blog
mysql -uroot -p123456
mysql> create database bbs; mysql> create database cms; mysql> create database blog; mysql> show databases; ##查看数据库 mysql> grant select,insert,update,delete,alter,create on cms.* to cms@"192.168.190.%" identified by '123456'; mysql> grant select,insert,update,delete,alter,create on bbs.* to bbs@'192.168.190.%' identified by '123456'; mysql> grant select,insert,update,delete,alter,create on blog.* to blog@'192.168.190.%' identified by '123456'; mysql> select user,host from mysql.user; ##查看用户是否创建成功
(3)MySQL数据库数据备份，利用周期性计划任务定时推送(简单的备份使用mysqldump，高层的备份服务使用replication或者drbd)
[root@mysql scripts]# cat mysqldunmp.sh
!#bin/bash cd /backup echo "You are in backup dir" File = /backup/ mysqldump -uroot -p123456 --quick --databases bbs blog cms --flush-logs --single-transaction > /backup/mysql$(date +%F).bak rsync -az /backup/mysql* [email protected]::mysql --password-file=/etc/rsync.password echo "Your database backup successfully completed"
vim /etc/rsync.password
123456
chmod 600 /etc/rsync.password
crontab -e
#send mysqlbak 00 00 * * * /bin/sh -x /server/script/mysqldump.sh
5.backup服务器(192.168.190.40 rsync ，每天检查推送过来的备份内容，定时发送邮件告知系统管理员备份是否成功)
useradd -s /sbin/nologin rsync 创建rsync 用户
yum install -y rsync
echo "/usr/bin/rsync --daemon" >> /etc/rc.local
vim /etc/rsyncd.conf
##rsyncd.conf start## uid = rsync gid = rsync use chroot = no max connections = 200 timeout = 300 pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock log file = /var/log/rsyncd.log ignore errors read only = false list = false hosts allow = 10.0.0.0/24 hosts deny = 0.0.0.0/32 auth users = rsync_backup secrets file = /etc/rsync.password [zhang] path = /zhang/ [cms] path =/data0/www/cms/ [bbs] path =/data0/www/bbs/ [blog] path =/data0/www/blog/ [backup] path =/backup/ [nfs] path =/backup/nfs/ [mysql] path =/mysql/
所有的推送文件夹必须存在 ，否则rsync启动会报错
chown -R rsync.rsync /zhang/
chown -R rsync.rsync /data0/www/cms/
chown -R rsync.rsync /data0/www/bbs/
chown -R rsync.rsync /data0/www/blog/
chown -R rsync.rsync /backup/
chown -R rsync.rsync /backup/nfs/
chown -R rsync.rsync /mysql/
vim /etc/rsync.password
rsync_backup:123456
chmod 600 /etc/rsync.password
echo “/usr/bin/rsync --daemon” >> /etc/rc.local
vim /server/script/check.sh
ls /mysql >> /root/check.txt
ls -l /backup/192.168.190.30/|awk '{print $9}' >> /root/check.txt
egrep -v "^$| " /root/check.txt > /root/checkadd.txt
最后推送checkadd.txt文本文档给系统管理员 管理员就可以看到都备份了什么文件
mail -s "Hello from linuxde.net by file" [email protected] < checkadd.txt
6.主nginx负载均衡服务器(192.168.190.23)
(1)安装配置nginx负载均衡器
编译nginx前 需安装
1.pcre pcre-devel
yum install -y pcre pcre-devel
2.openssl
yum install openssl openssl-devel -y
##创建nginx的虚拟用户
usedadd nginx -s /sbin/nologin -M
编译安装nginx
./configure --user=nginx --group=nginx --prefix=/application/nginx1.6.2 --with-http_stub_status_module --with-http_ssl_module make && make install ln -s /application/nginx1.6.2/ /application/nginx
##启动nginx
/application/nginx/sbin/nginx
vim /application/nginx/conf/nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream backend {
ip_hash;
server 192.168.190.10:80 max_fails=3 fail_timeout=30s;
server 192.168.190.20:80 max_fails=3 fail_timeout=30s;
}
upstream backendyy {
server 192.168.190.10:800 max_fails=3 fail_timeout=30s;
server 192.168.190.20:800 max_fails=3 fail_timeout=30s;
}
upstream backendblog {
ip_hash;
server 192.168.190.10:8000 max_fails=3 fail_timeout=30s;
server 192.168.190.20:8000 max_fails=3 fail_timeout=30s;
}
server {
listen 80;
server_name cms.etiantian.org;
index index.html index.htm;
location / {
proxy_pass http://backend;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
server {
listen 80;
server_name bbs.etiantian.org;
index index.html index.htm;
location / {
proxy_pass http://backendyy;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
server {
listen 80;
server_name blog.etiantian.org;
index index.html index.htm;
location / {
proxy_pass http://backendblog;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
平滑重启nginx ：/application/nginx/bin/nginx -s reload
(2)安装配置keepalived
yum install -y keepalived
netstat -lntup |grep keepalived
vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id lb01
}
vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 55
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
192.168.190.23/24 dev eth1 label eth1:1
}
}
/etc/init.d/keepalived restart ifconfig会发现生成了一个你想要的虚拟IP地址
7.高可用的备份nginx负载均衡服务器(192.168.190.23)
(1)安装配置nginx负载均衡器
编译nginx前 需安装
1.pcre pcre-devel
yum install -y pcre pcre-devel
2.openssl
yum install openssl openssl-devel -y
编译安装nginx
./configure --user=nginx --group=nginx --prefix=/application/nginx1.6.2 --with-http_stub_status_module --with-http_ssl_module make && make install ln -s /application/nginx1.6.2/ /application/nginx
##创建nginx的虚拟用户
usedadd nginx -s /sbin/nologin -M
##启动nginx
/application/nginx/sbin/nginx
vim /application/nginx/conf/nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream backend {
ip_hash;
server 192.168.190.10:80 max_fails=3 fail_timeout=30s;
server 192.168.190.20:80 max_fails=3 fail_timeout=30s;
}
upstream backendyy {
server 192.168.190.10:800 max_fails=3 fail_timeout=30s;
server 192.168.190.20:800 max_fails=3 fail_timeout=30s;
}
upstream backendblog {
ip_hash;
server 192.168.190.10:8000 max_fails=3 fail_timeout=30s;
server 192.168.190.20:8000 max_fails=3 fail_timeout=30s;
}
server {
listen 80;
server_name cms.etiantian.org;
index index.html index.htm;
location / {
proxy_pass http://backend;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
server {
listen 80;
server_name bbs.etiantian.org;
index index.html index.htm;
location / {
proxy_pass http://backendyy;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
server {
listen 80;
server_name blog.etiantian.org;
index index.html index.htm;
location / {
proxy_pass http://backendblog;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
平滑重启nginx ：/application/nginx/bin/nginx -s reload
(2)安装keepalived
yum install -y keepalived
vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface eth2
virtual_router_id 55
priority 100 ##优先级 数值越高越优先
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
192.168.190.23 dev eth2 label eth2:1
}
}
/etc/init.d/keepalived restart 启动keepalived 把主nginx负载均衡服务器宕掉 ，会发现备keepalived节点服务器会生成一个虚拟IP
本地做192.168.190.23的host解析
Keepalived高可用故障切换转移原理
Keepalived高可用服务对之间的故障切换转移，是通过VRRP来实现的。在keepalived服务工作时，主Master节点会不断地向备节点发送（多播的方式）心跳消息，用来告诉备Backup节点自己还活着。当主节点发生故障时，就无法发送心跳的消息了，备节点也因此无法继续检测到来自主节点的心跳了。于是就会调用自身的接管程序，接管主节点的IP资源和服务。当主节点恢复时，备节点又会释放主节点故障时自身接管的IP资源和服务，恢复到原来的备用角色