lnmap实战之负载均衡架构+高可用keepalived
lnmap实战之负载均衡架构(无高可用)
架构图如下:
此次实战软件,全部yum安装
1.准备好机器,同步好时间
192.168.42.150 node1 [负载均衡器]
192.168.42.152 node3 [web2]
192.168.42.153 node4 [web1]
192.168.42.151 node2 [memcached session存储]
192.168.42.154 node5 [nfs 共享存储]
192.168.42.155 node6 [mariadb]
我们这次实战从后面的节点开始吧
2.在node6节点上安装mariadb
安装mariadb
yum install mariadb-server -y
更改基本配置
vim /etc/my.cnf.d/server.cnf [mysqld] skip_name_resolve=1 log-bin=mysql-bin innodb_file_per_table = 1
启动mariadb
systemctl start mariadb
更改密码
mysqladmin -uroot -p password "root"
登录mysql
mysql -u root -p
添加外部访问账号
MariaDB [(none)]> grant all privileges on *.* to 'magedu'@'192.168.42.%' identified by '123456'; MariaDB [(none)]> flush privileges;
登录测试一下
mysql -u magedu -h 192.168.42.155 -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 7 Server version: 5.5.52-MariaDB MariaDB Server Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]>
3.node5 安装nfs
安装nfs
yum install nfs-utils rpcbind -y
添加用户(用户id,组id来自node3,node4的apache用户,我们需要统一)
[root@node5 ~] groupadd -g 48 apache [root@node5 ~] useradd -r -g 48 -u 48 apache [root@node5 ~] id apache uid=48(apache) gid=48(apache) groups=48(apache)
创建共享目录
mkdir -p /data chown -R apache.apache /data
配置共享目录
vim /etc/exports /data 192.168.42.0/24(rw,sync,all_squash,anonuid=48,anongid=48)
启动rpcbind,nfs
systemctl start rpcbind systemctl start nfs
设置开机启动 因为需要先启动rpcbind,然后再启动nfs,因此我们把开机启动放入rc.local里
chmod +x /etc/rc.local/rc.local vim /etc/rc.d/rc.local systemctl start rpcbind systemctl start nfs
查看此时的端口
ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:9743 *:* LISTEN 0 128 *:111 *:* LISTEN 0 128 *:20048 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 64 *:2049 *:* LISTEN 0 64 *:28518 *:* LISTEN 0 64 :::8812 :::* LISTEN 0 128 :::61836 :::* LISTEN 0 128 :::111 :::* LISTEN 0 128 :::20048 :::* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* LISTEN 0 64 :::2049 :::*
查看挂载的目录
showmount -e 127.0.0.1 Export list for 127.0.0.1: /application/data 168.92.168.42.0/24
去node4,node3挂载试一下
yum install nfs-utils rpcbind httpd -y id apache uid=48(apache) gid=48(apache) groups=48(apache) mkdir /test chown -R apache.apache /test mount -t nfs 192.168.42.154:/data /test
4.node4,node3在测试nfs的时候已经安装过httpd了 所以我们只需要在node4,node3节点上安装php就行了
yum install php php-mysql -y 此步骤只适合2.2版本,yum安装的2.4版本/etc/httpd/conf.d/php.conf已经帮我们添加了 配置apache虚拟主机,并让apache支持php vim /etc/httpd/conf/httpd.conf 找到对应的位置,添加 ServerName localhost:80 找到对应的位置,添加 LoadModule php5_module modules/libphp5.so 找到对应的位置,添加 AddType application/x-httpd-php .php 找到对应的位置,添加DirectoryIndex index.php index.html
测试php是否能顺利运行
vim /var/www/html/index.php
测试是否能顺利连接mysql
5.在node2上安装memcached
yum install libevent libevent-devle memcached -y systemctl enable memcached systemctl start memcached [root@node2 ~] ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 1024 *:11211 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* LISTEN 0 1024 :::11211 :::*
6.在node4,node3上操作,将php的session存储设置为memcached
(1)安装memcached扩展
yum install php-pecl-memcache* -y
(2)设置php的session存储
vim /etc/httpd/conf.d/php.conf php_value session.save_handler "memcache" php_value session.save_path "tcp://192.168.42.151:11211"
(3)用例子测试一下：
代码如下:
vim /var/www/html/session.php
(4).用另一个文件查看一下：
代码如下:
vim /var/www/html/get.php
7.在node1上安装nginx 负载均衡器
yum install nginx -y
vim /etc/nginx/conf.d/test.conf
server {
listen 80;
server_name www.test.com;
location / {
proxy_pass http://sshsrvs;
add_header X-Via $server_addr;
add_header X-Accel $server_name;
}
}
负载均衡配置
vim /etc/nginx/nginx.conf
upstream sshsrvs {
server 192.168.42.152:80;
server 192.168.42.153:80;
least_conn;
}
8.在node4,node3添加虚拟主机测试
node3:
创建应用目录 mkdir -p /application/discuz chown -R apache.apache /application/discuz 测试页 echo "this is discuz_1 test page." > /application/discuz/index.html 定义discuz虚拟主机 vim /etc/httpd/conf.d/discuz.confServerName test.discuz.com DocumentRoot "/application/discuz" 域名解析 vim /etc/hosts 192.168.42.152 test.discuz.com 重启apache systemctl restart httpdOptions None AllowOverride None Require all granted CustomLog "logs/iounix_access_log" combined
node4:
创建应用目录 mkdir -p /application/discuz chown -R apache.apache /application/discuz 测试页 echo "this is discuz_4 test page." > /application/discuz/index.html 定义discuz虚拟主机 vim /etc/httpd/conf.d/discuz.confServerName test.discuz.com DocumentRoot "/application/discuz" 域名解析 vim /etc/hosts 192.168.42.153 test.discuz.com 重启apache systemctl restart httpdOptions None AllowOverride None Require all granted CustomLog "logs/iounix_access_log" combined
9.在负载均衡上添加域名解析并测试
vim /etc/hosts 192.168.42.150 test.discuz.com
添加discuz虚拟主机
vim /etc/nginx/conf.d/discuz.conf
server {
listen 80;
server_name test.discuz.com;
location / {
proxy_pass http://sshsrvs;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
add_header X-Via $server_addr;
add_header X-Accel $server_name;
}
}
测试test.discuz.com
[root@node1 conf.d] for i in {1..10};do curl test.discuz.com ; done
this is discuz_1 test page.
this is discuz_4 test page.
this is discuz_1 test page.
this is discuz_4 test page.
this is discuz_1 test page.
this is discuz_4 test page.
this is discuz_1 test page.
this is discuz_4 test page.
this is discuz_1 test page.
this is discuz_4 test page.
负载均衡效果已经出来了
10.安装discuz测试
为了避免出错发生,我们需要把负载均衡,卸掉一个,留一个,因为我们是从物理机的浏览器安装discuz
在node1上注释掉一个
vim /etc/nginx/nginx.conf
upstream sshsrvs {
server 192.168.42.152:80;
server 192.168.42.153:80;
least_conn;
}
重启nginx
systemctl restart nginx
测试一下
[root@node1 nginx] for i in {1..10};do curl test.discuz.com ; done
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
this is discuz_1 test page.
进入node3的/application/discuz目录下载discuz
wget -c http://download.comsenz.com/DiscuzX/3.3/Discuz_X3.3_SC_UTF8.zip yum install unzip -y unzip Discuz_X3.3_SC_UTF8.zip ls [root@node3 discuz] ls Discuz_X3.3_SC_UTF8.zip index.html readme upload utility
删掉之前测试的index.html
rm -f index.html
我们可以看到discuz的目录层次有点深,因此我们需要把网站的根目录指向upload
vim /etc/httpd/conf.d/discuz.conf DocumentRoot "/application/discuz/upload"
重启apache
systemctl restart httpd
因为discuz需要检查目录权限,因此我们
chown -R apache.apache /application/discuz
添加物理机hosts域名解析
192.168.42.150 test.discuz.com
浏览器输入 test.discuz.com
可以看到以下效果:
按照提示操作即可完成discuz安装
11.discuz安装完成以后,我们需要把文件推送到node4节点上
cd /application scp -rp discuz [email protected]:/application
推送完成以后
node4操作:
cd /application/discuz rm -f index.html chown -R apache.apache /application/discuz
更改虚拟主机discuz的根目录为upload
vim /etc/httpd/conf.d/discuz.conf DocumentRoot "/application/discuz/upload"
完成上述操作以后,还记得之前在node1上注释的吗,我们需要把它打开
vim /etc/nginx/nginx.conf
upstream sshsrvs {
server 192.168.42.152:80;
server 192.168.42.153:80;
least_conn;
}
重启nginx
12.在去浏览器中访问,登录后台,操作等,一切正常
13.还有最后一个问题需要处理,discuz的数据目录,共享存储 在node5上操作:
mkdir -p /data/discuz/
进入node3 把data推送过来
cd /application/discuz/upload scp -rp uc_server [email protected]:/data/discuz/ scp -rp data [email protected]:/data/discuz/
在node5上操作:
vim /etc/exports /data/discuz/data 192.168.42.0/24(rw,sync,all_squash,anonuid=48,anongid=48) /data/discuz/uc_server 192.168.42.0/24(rw,sync,all_squash,anonuid=48,anongid=48) exportfs -rv chown -R apache.apache /data
在node3上挂载并加入开机挂载
cd /application/discuz/upload/data rm -rf * umount /test cd ../ mount -t nfs 192.168.42.154:/data/discuz/data /application/discuz/upload/data mount -t nfs 192.168.42.154:/data/discuz/uc_server /application/discuz/upload/uc_server chmod +x /etc/rc.d/rc.local echo "mount -t nfs 192.168.42.154:/data/discuz/data /application/discuz/upload/data" >>/etc/rc.d/rc.local echo "mount -t nfs 192.168.42.154:/data/discuz/uc_server /application/discuz/upload/uc_server" >>/etc/rc.d/rc.local
在node4做同样的操作:
至此工作已经全部完成
添加高可用
“lnmap实战之负载均衡架构(无高可用)”之新增keepalived高可用
我之前有一篇”lnmap实战之负载均衡架构(无高可用)“博客是专门部署了lanmap,之前没有做高可用,那么我们现在就把高可用补上去吧
这样我们照着之前的文档从新部署一下 1.机器结构如下:
192.168.42.150 node0 [负载均衡器] 192.168.42.151 node1 [负载均衡器 新增] 192.168.42.152 node2 [web1] 192.168.42.153 node3 [web2] 192.168.42.154 node4 [nfs 共享存储] 192.168.42.155 node5 [memcached session存储] 192.168.42.156 node6 [mariadb]
我们在此基础之上新增一台nginx[负载均衡器]
我们此次是针对这两台nginx负载均衡器做高可用
2.我们依照上面的机器结构和之前的部署文档，把”lnmap实战之负载均衡架构(无高可用)”部署出来，部署出来，测试没问题之后我们在node1节点上安装nignx
安装方法和node0一样
node1操作：
(1).安装nginx
yum install nginx -y
(2).备份nginx.conf
cp /etc/nginx/nginx.conf{,.back}
(3).将node0的nginx配置文件cp一份到node1 ->[记得是在node0上操作]
scp /etc/nginx/nginx.conf 192.168.42.151:/etc/nginx/ cd /etc/nginx/conf.d/ scp *.conf 192.168.42.151:/etc/nginx/conf.d/
(4).更改node1上的hosts文件
echo "192.168.42.151 www.test.com">>/etc/hosts echo "192.168.42.151 test.discuz.com">>/etc/hosts cat /etc/hosts nginx -t nginx -s reload
3.nginx+keepalived高可用
node0,node1 关闭nignx
node0:
(1).安装keepalived
yum install keepalived -y
(2).配置keepalived
配置如下：
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node0
vrrp_mcast_group4 224.1.101.23
}
#存在文件时，检测成功，即执行降级；否则不存在,全部退出；实现服务器切换
vrrp_script chk_down{
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
interval 1
weight -10
fall 1
rize 1
}
#脚本，健康状态检测，检测nginx是否存活
vrrp_script chk_nginx {
script "killall -0 nginx && exit 0 || exit 1"
interval 1
weight -10
fall 1
rise 1
}
vrrp_instance sr1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass rEiszbuO
}
virtual_ipaddress {
192.168.42.182/24 dev ens33 label ens33:0
}
#脚本调用
track_script {
chk_down
chk_nginx
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
node1:
(1).安装keepalived
yum install keepalived -y
(2).配置keepalived
配置如下：
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1
vrrp_mcast_group4 224.1.101.23
}
#存在文件时，检测成功，即执行降级；否则不存在,全部退出；实现服务器切换
vrrp_script chk_down{
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
interval 1
weight -10
fall 1
rize 1
}
#脚本，健康状态检测，检测nginx是否存活
vrrp_script chk_nginx {
script "killall -0 nginx && exit 0 || exit 1"
interval 1
weight -10
fall 1
rise 1
}
vrrp_instance sr1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 96
advert_int 1
authentication {
auth_type PASS
auth_pass rEiszbuO
}
virtual_ipaddress {
192.168.42.182/24 dev ens33 label ens33:0
}
#脚本调用
track_script {
chk_down
chk_nginx
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
调用脚本notify.sh：
cd /etc/keepalived/
[root@centos703 keepalived]# vim notify.sh
#!/bin/bash
#
contact='root@localhost'
notify() {
mailsubject="vrrp:$(hostname) to be $1"
mailbody="$(hostname) to be $1,vrrp transition, $(date)."
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
systemctl start nginx
;;
backup)
notify backup
systemctl start nginx
;;
fault)
notify fault
systemctl stop nginx
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
6.node0,node1重新更改hosts解析
vim /etc/hosts 192.168.42.182 www.test.com 192.168.42.182 test.discuz.com
7.我们在node3,node4上是做了两个虚拟主机，因此我们在负载均衡上也是要做两个虚拟主机，并且需要把$host推送到后方的机器
test.conf 的内容如下：
server {
listen 80;
server_name www.test.com;
location / {
proxy_pass http://sshsrvs;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
add_header X-Via $server_addr;
add_header X-Accel $server_name;
}
}
discuz.conf的内容如下：
server {
listen 80;
server_name test.discuz.com;
location / {
proxy_pass http://sshsrvs;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
add_header X-Via $server_addr;
add_header X-Accel $server_name;
}
}
8.我们在浏览器中访问www.test.com ,test.discuz.com
是两个不同的网站记得在物理机需要做域名解析哦