Haproxy+keepalived高可用集群

LNMP高可用集群架构

• Haproxy+keeplived :192.168.15.105,192.168.15.106
• Nginx+php :192.168.15.107,192.168.15.108
• NFS+rsyncd :192.168.15.111,192.168.15.112
• mysql :192.168.15.114

一，安装haproxy+keepalived

1，安装配置haproxy

IP105，IP106 两台主机上编译安装haprxoy

1) 编译安装HAProxy

• yum install gcc gcc-c++ glibc glibc-devel pcre pcre-devel openssl openssl-devel systemd-devel net-tools vim iotop bc zip unzip zlib-devel lrzsz tree screen lsof tcpdump wget ntpdate
• cd /usr/local/src/
• tar xvf haproxy-1.8.20.tar.gz && cd haproxy-1.8.20
• make ARCH=x86_64 TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1 USE_CPU_AFFINITY=1 PREFIX=/usr/local/haproxy
• make install PREFIX=/usr/local/haproxycp
• cp haproxy /usr/sbin/

2)创建启动脚本

• vim /usr/lib/systemd/system/haproxy.service
• [Unit]
• Description=HAProxy Load Balancer
• After=syslog.target network.target
• [Service]
• ExecStartPre=/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c -q
• ExecStart=/usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
• ExecReload=/bin/kill -USR2 $MAINPID
• [Install]
• WantedBy=multi-user.target

3)创建目录和用户

• mkdir /etc/haproxy
• vim /etc/haproxy/haproxy.cfg
• global
• maxconn 100000
• chroot /usr/local/haproxy
• #stats socket /var/lib/haproxy/haproxy.sock mode 600 level admin
• uid 99
• gid 99
• daemon
• nbproc 2
• nbthread 2
• cpu-map 1 0
• cpu-map 2 1
• #cpu-map 3 2
• #cpu-map 4 3
• pidfile /run/haproxy.pid
• log 127.0.0.1 local3 info
• defaults
• option http-keep-alive
• option forwardfor
• maxconn 100000
• mode http
• timeout connect 300000ms
• timeout client 300000ms
• timeout server 300000ms
• listen stats
• mode http
• bind 0.0.0.0:9999
• stats enable
• log global
• stats uri /haproxy-status
• stats auth haadmin:q1w2e3r4ys
• listen web_port
• bind 0.0.0.0:80
• mode http
• log global #
• server web1 127.0.0.1:8080 check inter 3000 fall 2 rise 5
• #wordpress访问入口======================================
• listen WEB_PORT_80
• bind 192.168.15.240:80
• mode http
• balance source
• server web1 192.168.15.107:80 weight 1 check inter 3000 fall 3 rise 5
• server web2 192.168.15.108:80 weight 1 check inter 3000 fall 3 rise 5
• #数据库访问入口======================================
• listen mysql_port
• bind 192.168.15.240:3306
• mode tcp
• server web1 192.168.15.114:3306 check inter 3000 fall 3 rise 5

添加haproxy用户创建相关目录授权

• useradd haproxy -s /sbin/nologin
• mkdir /var/lib/haproxy
• chown haproxy.haproxy /var/lib/haproxy/ -R
• mkdir -p /usr/local/haproxy/run/

4)启动HAProxy

• systemctl enable haproxy
• systemctl restart haproxy

2，安装配置keepalived

• yum install keepalived ipvsadm -y

1)配置IP105

• vim /etc/keepalived/keepalived.conf
• vrrp_iptables
• vrrp_garp_interval 0
• vrrp_gna_interval 0
• }
• vrrp_instance VIP1 {
• state MASTER
• interface ens33
• virtual_router_id 51
• priority 100
• advert_int 2
• authentication {
• auth_type PASS
• auth_pass centos
• }
• virtual_ipaddress {
• 192.168.15.240 dev ens33 label ens33:0
• }
• }

2)配置IP106

• vim /etc/keepalived/keepalived.conf
• vrrp_iptables
• vrrp_garp_interval 0
• vrrp_gna_interval 0
• }
• vrrp_instance VIP1 {
• state MASTER
• interface ens33
• virtual_router_id 51
• priority 80
• advert_int 2
• authentication {
• auth_type PASS
• auth_pass centos
• }
• virtual_ipaddress {
• 192.168.15.240 dev ens33 label ens33:0
• }
• }

3)配置默认参数

• vim /etc/sysctl.conf
• net.ipv4.ip_nonlocal_bind = 1
• net.ipv4.ip_forward = 1

sysctl -p

二,部署nginx+php

IP107,IP108部署nginx+php
nginx,php用相同用户www启动

1，部署nginx

1）源码编译安装nginx

• yum install -y vim lrzsz tree screen psmisc lsof tcpdump wget ntpdate gcc gcc-c++ glibc glibc-devel pcre pcre-devel openssl openssl-devel systemd-devel net-tools iotop bc zip unzip zlib-devel bash-completion nfs-utils automake libxml2 libxml2-devel libxslt libxslt-devel perl perl-ExtUtils-Embed
• cd /usr/local/src/
• wget http://nginx.org/download/nginx-1.14.2.tar.gz
• tar -xf nginx-1.14.2.tar.gz
• cd nginx-1.14.2
• ./configure --prefix=/apps/nginx \
• --user=www \
• --group=www \
• --with-http_ssl_module \
• --with-http_v2_module \
• --with-http_realip_module \
• --with-http_stub_status_module \
• --with-http_gzip_static_module \
• --with-pcre \
• --with-stream \
• --with-stream_ssl_module \
• --with-stream_realip_module
• make && make install
• chown www.www -R /apps/nginx/

2）创建Nginx自启动脚本

• vim /usr/lib/systemd/system/nginx.service
• [Unit]
• Description=nginx - high performance web server
• Documentation=http://nginx.org/en/docs/
• After=network-online.target remote-fs.target nss-lookup.target
• Wants=network-online.target
• [Service]
• Type=forking
• PIDFile=/apps/nginx/logs/nginx.pid
• ExecStart=/apps/nginx/sbin/nginx -c /apps/nginx/conf/nginx.conf
• ExecReload=/bin/kill -s HUP $MAINPID
• ExecStop=/bin/kill -s TERM $MAINPID
• [Install]
• [Install]
• WantedBy=multi-user.target
• ln -sv /apps/nginx/sbin/nginx /usr/sbin/

3）配置启动nginx

• user www www;
• worker_processes auto;
• pid logs/nginx.pid;
• include /apps/nginx/conf/server/*.conf;
• }
• mkidr -p /apps/nginx/conf/server/
• vim /apps/nginx/conf/server/wordpress.conf
• server {
• listen 80;
• server_name www.wordpress.net;
• location / {
• root /data/nginx/wordpress;
• index index.php index.html index.htm;
• }
• location ~ .php$ {
• root /data/nginx/wordpress;
• fastcgi_pass 127.0.0.1:9000
• fastcgi_index index.php;
• fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
• include fastcgi_params;
• }
• }

2，部署php 7.3.5

1）编译安装php 7.3.5

• cd /usr/local/src
• yum -y install wget vim pcre pcre-devel openssl openssl-devel libicu-devel gcc gcc-c++ autoconf libjpeg libjpeg-devel libpng libpng-devel freetype\
• freetype-devel libxml2 libxml2-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel ncurses ncurses-devel curl curl-devel krb5-devel libidn libidn-devel openldap\
• openldap-devel nss_ldap jemalloc-devel cmake boost-devel bison automake libevent libevent-devel gd gd-devel libtool* libmcrypt libmcrypt-devel mcrypt mhash libxslt\
• libxslt-devel readline readline-devel gmp gmp-devel libcurl libcurl-devel openjpeg-devel
• tar xvf php-7.3.5.tar.bz2
• yum -y install libxml2-devel bzip2-devel libmcrypt-devel (基于EPEL)
• cd php-7.3.5/
• ./configure --prefix=/apps/php \
• --enable-mysqlnd \
• --with-mysqli=mysqlnd \
• --with-pdo-mysql=mysqlnd \
• --with-openssl \
• --with-freetype-dir \
• --with-jpeg-dir \
• --with-png-dir \
• --with-zlib \
• --with-libxml-dir=/usr \
• --with-config-file-path=/etc \
• --with-config-file-scan-dir=/etc/php.d \
• --enable-mbstring \
• --enable-xml \
• --enable-sockets \
• --enable-fpm \
• --enable-maintainer-zts \
• --disable-fileinfo
• make && make install

2）配置php

• cd /apps/php/etc/php-fpm.d/
• cp www.conf.default www.conf
• cp /usr/local/src/php-7.3.5/php.ini-production /apps/php/etc/php.ini
• useradd www -s /sbin/nologin -u 1001
• vim www.conf
• user = www
• group = www
• listen = 127.0.0.1:9000
• /apps/php/sbin/php-fpm -t
• /apps/php/sbin/php-fpm -c /apps/php/etc/php.ini
• ps -ef | grep php-fpm

3）准备PHP测试页主机本地解析测试

• mkdir /data/nginx/wordpress -p
• vim /data/nginx/wordpress/index.php
• phpinfo();
• ?>

三，部署mysql

IP104 192.168.15.104

脚本部署mysql

• cd /usr/local/src
• tar xf mysql-5.6.34-onekey-install.tar.gz
• bash mysql-install.sh
• mysql
• CREATE DATABASE wordpress;
• GRANT ALL PRIVILEGES ON wordpress.* TO "wordpress"@"192.168.15.%" IDENTIFIED BY "centos";

四,部署NFS+rsyncd

IP111 192.168.15.111
IP112 192.168.15.112

1）部署NFS

IP111 192.168.15.111

安装NFS

• yum install nfs-utils -y
• mkdir /nfsdata/wordpress -p
• vim /etc/exports
• /nfsdata/wordpress *(rw,no_root_squash)

2）配置nfs实时备份

在nfs和nfsbak服务器上安装rsync

• yum -y install rsync

在NFSbak端配置为rsync服务

• mkdir /data/backup
• vim /etc/rsyncd.conf
• uid = root
• gid = root
• use chroot = no
• max connections = 0
• log file = /var/log/rsyncd.log
• pid file = /var/run/rsyncd.pid
• reverse lookup = no
• host allow = 192.168.15.111
• [backup]
• path = /data/backup/
• comment = backup
• readonly = no
• auth users = rsyncuser
• secrets file = /etc/rsync.pass

生成验证用的账户密码，修改为安全权限

• echo "rsyncuser:centos" > /etc/rsync.pass
• chmod 600 /etc/rsync.pass

启动服务

• systemctl start rsyncd

NFS端配置,测试rsync一次性任务

• yum install inotify-tools -y
• echo "centos" > /etc/rsync.pass
• chmod 600 /etc/rsync.pass
• touch /data/html/f1
• rsync -avz --password-file=/etc/rsync.pass /nfsdata/wordpress/ [email protected]::backup

让inotify配合sync实时同步

在nfs端创建脚本

• vim rsync.sh
• #!/bin/bash
• SRC='/nfsdata/wordpress/'
• DEST='[email protected]::backup'
• inotifywait -mrq --timefmt '%Y-%m-%d %H:%M' --format '%T %w %f' -e create,delete,moved_to,close_write,attrib ${SRC} |while read DATE TIME DIR FILE;do
• FILEPATH=${DIR}${FILE}
• rsync -az --delete --password-file=/etc/rsync.pass $SRC $DEST && echo "At ${TIME} on ${DATE}, file $FILEPATH was backuped up via rsync" >> /var/log/changelist.log
• done
• bash rsync.sh

五,安装wordpress

IP107 192.168.15.107
IP108 192.168.15.108

1）配置IP107 192.168.15.107

修改本机/etc/hosts 解析mysqlvip.com 192.168.15.240

• cd /data/nginx/wordpress
• tar -xf wordpress-5.0.1-zh_CN.tar.gz
• mv wordpress-5.0.1-zh_CN.tar.gz /opt
• mv wordpress/* .
• mv wordpress/ /opt
• cp wp-config-sample.php wp-config.php
• vim wp-config.php 配置通过haproxy连接数据库
• chown -R www.www /data/nginx/wordpress

2)配置IP108 192.168.15.108,同IP107

scp -r ./* 192.168.15.108:/data/nginx/wordpress

六，实现NFS挂载

IP107
在本地浏览器中完成wordpress的安装配置上传图片

• cd /data/nginx/wordpress/wp-content/uploads/
• mv 2019 /opt

将nfs开机挂载
IP107 192.168.15.107
IP108 192.168.15.108

• vim /etc/fstab
• 192.168.15.111:/nfsdata/wordpress /data/nginx/wordpress/wp-content/uploads nfs defaults,_netdev 0 0
• cd
• mount -a
• df
• mv /opt/2019 /data/nginx/wordpress/wp-content/uploads/

七，本地浏览器测试

• www.wordpress.net