Keepalived、varnish

1、Nginx+Keepalived实现站点高可用

  • 网络拓扑


    网络拓扑
  • 环境说明
    • Nginx代理服务器:10.192.1.163、10.192.1.164
    • Keepalived 高可用程序提供对外服务IP地址VIP:10.192.1.99
    • Web服务:10.192.1.66、10.192.1.67、10.192.1.68提供站点服务
  • 搭建
    • 10.192.1.163/64安装、配置Nginx和Keepalived
    安装程序包
    yum -y install nginx keepalived
    
    配置nginx代理
    vim /etc/nginx/nginx.conf
    http {
    ...
    upstream web {                     #用upstream模拟四层代理,对后端web站点进行代理
          server 10.192.1.66:80;  
          server 10.192.1.67:80;
          server 10.192.1.68:80;
          }
    server {
    ...
    location / {
                  proxy_pass http://web;   
          }
    ...
    }
    ...
    }
    
    配置keepalived (MASTER)
    vim keepalived.conf
    ! Configuration File for keepalived
    
     global_defs {
     notification_email {
          root@localhost
     }
     notification_email_from keepalived@localhost
     smtp_server 127.0.0.1
     smtp_connect_timeout 30
     router_id node1
     vrrp_mcast_group4 224.1.1.11
    }
    
    vrrp_script chk_down {
      script "/etc/keepalived/file.sh"      #文件检测脚本
      weight -5
      interval 1
      fall 1
      rise 1
    }
    
    vrrp_script chk_nginx {
      script "killall -0 nginx && exit 0 || exit 1"   #nginx进程检测脚本
      weight -5
      interval 2
      fall 3
      rise 3
    }
    
    vrrp_instance VI_1 {             #虚拟路由配置
      state MASTER
      interface ens32
      virtual_router_id 25
      priority 100
      advert_int 1
      authentication {
          auth_type PASS
          auth_pass 11112222
      }
      virtual_ipaddress {
          10.192.1.99/24 dev ens32 label ens32:0
      }
    
      track_script {         #监控脚本
      chk_down
      chk_nginx
          }
      notify_master       "/etc/keepalived/notify.sh master"      #转换通知
      notify_backup       "/etc/keepalived/notify.sh backup"
      notify_fault        "/etc/keepalived/notify.sh fault"
    }
    配置keepalived (BACKUP)
    router_id node2
    state BACKUP
    priority 98
    
    notify.sh脚本,
    #!/bin/bash
    #
    contact='root@localhost'
    
    notify() {
    local mailsubject="$(hostname) to be $1,vip floating"
    local mailbody="$(date +'%F %T'):vrrp transition,$(hostname) change to $1"
    echo "$mailbody" |mail -s "$mailsubject" $contact
    }
    
    case $1 in
    master)
          systemctl start nginx   #转变为master时开启nginx
          notify master
          ;;
    backup)
          systemctl stop nginx   #转变为bakcup时停止nginx
          notify backup
          ;;
    fault)
          systemctl stop nginx   #转变为失败时停止nginx
          notify fault
          ;;
    *)
          echo "Usage:$(basename $0) {master|backup|fault}"
          exit 1
          ;;
    esac
    
    file.sh 手动切换脚本    存在down文件就发生切换
    #!/bin/bash
    #
    [[ -f /etc/keepalived/down ]] && exit 1 || exit 0
    
    
    • 10.192.1.66/67/68安装、配置web服务,这里也安装Nginx
    10.192.1.66/67/68 提供同一个web站点
    为演示实现效果这里分别用
    

    web 66

    web 67

    web 66

    来表示不同的机器

  • 测试可用性
    • 模拟后台66宕机
    在66主机上执行iptables命令
    [root@localhost conf.d]# iptables -I INPUT -d 10.192.1.66 -j REJECT
    
    客户机每3秒访问一次站点
    [root@ceph_deploy ping]# for  i in {1..10} ;do sleep 3 ;curl 10.192.1.99 ;done
    

    web 66

    web 68

    web 67

    web 67

    web 68

    web 67

    web 68

    web 67

    web 68

    web 68

    [root@ceph_deploy ping]#

    • 模拟Nginx代理服务器故障和恢复
    在163主机上touch一个down文件,实现vip地址漂移
    [root@node1 keepalived]# touch down
    
    通过抓包监控主备切换
    20:03:20.487986 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:03:21.489032 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 95, authtype simple, intvl 1s, length 20
    20:03:21.517232 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20
    20:03:22.517422 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20
    20:03:23.519136 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20
    20:03:24.520448 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20
    
    在163主机上将down文件删除,实现vip地址漂移回来
    20:07:12.616295 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20
    20:07:13.617912 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20
    20:07:13.619004 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:07:14.618981 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:07:15.620928 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:07:16.624430 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    

2、实现keepalived主主模型

  • 网络拓扑


    网络拓扑
  • 环境说明
    • Nginx代理服务器:10.192.1.163、10.192.1.164
    • Keepalived 高可用程序提供对外服务IP地址VIP:10.192.1.98和10.192.1.99
    • Web服务:10.192.1.66、10.192.1.67、10.192.1.68提供站点服务
    • 10.192.1.98和10.192.1.99可以代理同一个web站点,也可以是不同站点,本实验为同一个站点
    • VIP:10.192.1.99以10.192.1.163为主,VIP:10.192.1.99以10.192.1.164为主
  • 搭建
    在上面配置的基础上只需要对keepalived配置进行响应修改即可
在163服务器上配置如下:
[root@node1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
    root@localhost
   }
   notification_email_from keepalived@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_mcast_group4 224.1.1.11
}

vrrp_script chk_down {
    script "/etc/keepalived/file.sh"
    weight -5
    interval 1
    fall 1
    rise 1
}

vrrp_script chk_nginx {
    script "killall -0 nginx && exit 0 || exit 1"
    weight -5
    interval 2
    fall 3
    rise 3
}

vrrp_instance VI_1 {
    state MASTER
    interface ens32
    virtual_router_id 25
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        10.192.1.99/24 dev ens32 label ens32:0
    }
    
    track_script {
    chk_down
    chk_nginx
    }
    notify_master       "/etc/keepalived/notify.sh master"
    notify_backup       "/etc/keepalived/notify.sh backup"
    notify_fault        "/etc/keepalived/notify.sh fault"
}

vrrp_instance VI_2 {   #添加第二个虚拟路由器配置
    state BACKUP
    interface ens32
    virtual_router_id 24
    priority 98
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        10.192.1.98/24 dev ens32 label ens32:1
    }
   track_script {
    chk_down
    chk_nginx
        }

    notify_master       "/etc/keepalived/notify.sh master"
    notify_backup       "/etc/keepalived/notify.sh backup"
    notify_fault        "/etc/keepalived/notify.sh fault"
}

在164服务器上配置
[root@node2 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
    root@localhost
   }
   notification_email_from keepalived@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   #vrrp_skip_check_adv_addr
   #vrrp_strict
   #vrrp_garp_interval 0
   #vrrp_gna_interval 0
   vrrp_mcast_group4 224.1.1.11
}

vrrp_script chk_down {
    script "/etc/keepalived/file.sh"
    weight -5
    interval 1
    fall 1
    rise 1
}

vrrp_script chk_nginx {
    script "killall -0 nginx && exit 0 || exit 1"
    weight -5
    interval 2
    fall 3
    rise 3
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens32
    virtual_router_id 25
    priority 98
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        10.192.1.99/24 dev ens32 label ens32:0
    }
    
    track_script {
    chk_down
    chk_nginx
    }
    notify_master       "/etc/keepalived/notify.sh master"
    notify_backup       "/etc/keepalived/notify.sh backup"
    notify_fault        "/etc/keepalived/notify.sh fault"
}

vrrp_instance VI_2 {  #第二个虚拟路由器配置
    state MASTER
    interface ens32
    virtual_router_id 24
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 11112222
    }
    virtual_ipaddress {
        10.192.1.98/24 dev ens32 label ens32:1
    }
   track_script {
    chk_down
    chk_nginx
        }

    notify_master       "/etc/keepalived/notify.sh master"
    notify_backup       "/etc/keepalived/notify.sh backup"
    notify_fault        "/etc/keepalived/notify.sh fault"
}

  • 测试
    • 正常情况下
    20:36:06.616278 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20
    20:36:06.618772 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:36:07.616765 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20
    20:36:07.618843 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:36:08.617185 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20
    20:36:08.619023 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:36:09.617493 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20
    20:36:09.620231 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    20:36:10.617977 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20
    20:36:10.619289 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20
    
    [root@ceph_deploy ping]# for  i in {1..10} ;do curl 10.192.1.99 && curl 10.192.1.98 ;echo "--------";done
    

    web 67

    web 67

    --------

    web 67

    web 68

    --------

    web 68

    web 66

    --------

    web 68

    web 66

    --------

    web 66

    web 68

    --------

    web 66

    web 67

    --------

    web 67

    web 67

    --------

    web 67

    web 68

    --------

    web 68

    web 66

    --------

    web 68

    web 66

    -------- [root@ceph_deploy ping]#

    • 模拟163主机宕机
    163 停止nginx服务
    [root@node1 keepalived]# systemctl stop nginx
    
    客户端访问站点情况
    [root@ceph_deploy ping]# for  i in {1..10} ;do sleep 3; curl 10.192.1.99 ; curl 10.192.1.98 ;echo "--------";done
    

    web 66

    web 67

    --------

    web 66

    web 66

    -------- curl: (7) Failed connect to 10.192.1.99:80; Connection refused

    web 68

    -------- curl: (7) Failed connect to 10.192.1.99:80; No route to host

    web 68

    --------

    web 67

    web 66

    --------

    web 66

    web 67

    --------

    web 68

    web 67

    --------

    web 66

    web 68

    --------

    web 67

    web 66

    --------

    web 68

    web 66

    -------- VIP飘移过程 20:45:55.961315 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20 20:45:55.974024 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20 20:45:56.961483 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 100, authtype simple, intvl 1s, length 20 20:45:56.974199 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20 20:45:57.961596 IP 10.192.1.163 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 95, authtype simple, intvl 1s, length 20 20:45:57.962084 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20 20:45:57.974398 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20 20:45:58.962709 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20 20:45:58.974582 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20 20:45:59.964495 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20 20:45:59.974764 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20 20:46:00.965066 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20 20:46:00.974976 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20 20:46:01.966199 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20 20:46:01.975185 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20 20:46:02.966805 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 25, prio 98, authtype simple, intvl 1s, length 20 20:46:02.975735 IP 10.192.1.164 > 224.1.1.11: VRRPv2, Advertisement, vrid 24, prio 100, authtype simple, intvl 1s, length 20

3、采用varnish为nginx实现缓存加速

  • 环境准备
    varnish服务器:10.192.1.167
    nginx服务器:10.192.1.99
  • varnish安装配置
[root@localhost ~]# yum -y install varnish
[root@localhost ~]# cd /etc/varnish/
[root@localhost varnish]# vim varnish.params
RELOAD_VCL=1
VARNISH_VCL_CONF=/etc/varnish/default.vcl
VARNISH_LISTEN_ADDRESS=10.192.1.167
VARNISH_LISTEN_PORT=80
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
VARNISH_SECRET_FILE=/etc/varnish/secret
VARNISH_STORAGE="file,/data/varnish/cache,256M"
VARNISH_USER=varnish
VARNISH_GROUP=varnish

[root@localhost varnish]# vim default.vcl 
backend default {
    .host = "10.192.1.99";
    .port = "80";
}
sub vcl_recv {
}
sub vcl_backend_response {
}
sub vcl_deliver {
 if (obj.hits>0) {
      set resp.http.X-Cache="HIT via " + server.ip;
  } else {
      set resp.http.X-Cache="MISS via " + server.ip;
  }
}

[root@localhost varnish]# systemctl enable varnish
[root@localhost varnish]# systemctl start varnish
  • 测试
[root@ceph_deploy ~]# curl -I http://10.192.1.167
HTTP/1.1 200 OK
Server: nginx/1.12.2
Date: Mon, 29 Apr 2019 12:08:58 GMT
Content-Type: text/html
Content-Length: 17
Last-Modified: Mon, 29 Apr 2019 06:20:10 GMT
ETag: "5cc6979a-11"
X-Varnish: 33065
Age: 0
Via: 1.1 varnish-v4
X-Cache: MISS via 10.192.1.167      第一次访问没有命中缓存
Connection: keep-alive

[root@ceph_deploy ~]# curl -I http://10.192.1.167
HTTP/1.1 200 OK
Server: nginx/1.12.2
Date: Mon, 29 Apr 2019 12:08:58 GMT
Content-Type: text/html
Content-Length: 17
Last-Modified: Mon, 29 Apr 2019 06:20:10 GMT
ETag: "5cc6979a-11"
X-Varnish: 290 33066
Age: 33
Via: 1.1 varnish-v4
X-Cache: HIT via 10.192.1.167     第二次访问命中缓存
Connection: keep-alive

[root@ceph_deploy ~]# 

4、LNMP结合varnish实现动静分离

  • 网络拓扑


    动静分离网络拓扑
  • 环境准备
    10.192.1.163 安装varnish
    10.192.1.161 安装Nginx提供web静态页面服务
    10.192.1.167 安装Nginx、php-fpm、mariadb提供动态站点服务
  • varnish服务器配置
修改varnish主配置文件将监听端口修改为80端口
[root@node1 varnish]# vim varnish.params
VARNISH_LISTEN_PORT=80
启动varinish服务
[root@node1 varnish]# systemctl  start varnish

修改varinish缓存规则文件
vcl 4.0;
backend default {                          #默认后端服务器
    .host = "10.192.1.167";      
    .port = "80";
}

backend nginxsrvs {               #静态网页web服务器
    .host = "10.192.1.161";
    .port = "80";
}

sub vcl_purge {                       #定义清除缓存规则
     return (synth(200,"Purged"));  
}

sub vcl_recv {
if (req.method == "PURGE") {          #通过PURGE方法清除缓存
    return (purge);
}

if (req.url ~ "(?i)^/(login|admin)") {
                                        return(pass);
                                }
if (req.url ~ "(?i)\.(html|htm|css|svg|js|jpg|jpeg|png|gif|pdf)") {         #访问静态内容转发到静态服务器
         set req.backend_hint = nginxsrvs;
    } else {
         set req.backend_hint =  default;            #否则转发至默认主机
    }

}


sub vcl_backend_response {

}

sub vcl_deliver {
if (obj.hits>0) {
      set resp.http.X-Cache="HIT via " + server.ip;
  } else {
      set resp.http.X-Cache="MISS via " + server.ip;
  }
}
       
使配置规则生效
[root@node1 varnish]# varnish_reload_vcl         
  • 配置nmp服务器
这里安装WordPress软件提供动态web站点,安装过程比较简单,官网下载,解压至web目录
这里是/data/wordpress目录
[root@localhost wordpress]#  tar -xzvf wordpress-5.0.3-zh_CN.tar.gz -C /data/

启动php-fpm服务
[root@localhost wordpress]# systemctl start php-fpm

启动mariadb创建数据库并授权访问
[root@localhost wordpress]# systemctl start mariadb
MariaDB [(none)]> CREATE DATABASE wordpress;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON wordpress.* TO 'wordpress'@'localhost' IDENTIFIED  BY 'wordpress';

配置nginx,并启动nginx服务
server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
        #root         /usr/share/nginx/html;
        root          /data/wordpress;
        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;
        index index.php index.html index.htm;
        location / {
        }
        location ~ \.php$ {
                root /data/wordpress;
                fastcgi_pass 127.0.0.1:9000;        
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                include fastcgi_params;
                }
}
[root@localhost wordpress]# systemctl  start nginx

wordpress url设置
WordPress地址(URL)    http://www.mywp.com

  • 配置静态web服务器
这个web服务器需要实时同步nmp服务器的WordPress目录,我们这里采用nfs方式
nmp服务器作为nfs服务端,静态web服务器作为nfs客户端,安装过程比较简单,如下命令
[root@localhost wordpress]# yum -y install rpcbind nfs-utils

静态服务器mount网络文件,同样放在/data/wordpress目录下
[root@ceph_deploy ~]# mount -t nfs 10.192.1.167:/data/wordpress /data/wordpress

配置nginx并启动
server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
        #root         /usr/share/nginx/html;
        root         /data/wordpress;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;
        index index.html index.htm;
        location / {
        }
[root@ceph_deploy ~]# systemctl  start nginx

编辑一个静态页面
[root@localhost wordpress]# vim index.html 
static pag
  • 测试
    客户端主机设置hosts
    10.192.1.163 www.mywp.com # 指向varinish服务器
    访问主页站点

    压力测试压文章
    [root@localhost wordpress]# ab -n 1000 -c 100 http://www.mywp.com/?p=10
    压力测试压静态页面
    [root@localhost wordpress]# ab -n 1000 -c 100 http://www.mywp.com/index.html
    查看varnish服务器状态
[root@node1 ~]# varnishstat -1 -l -f MAIN.cache_hit  -1 -l -f MAIN.cache_miss 
MAIN.cache_hit            5773         0.10 Cache hits
MAIN.cache_miss             95         0.00 Cache misses

你可能感兴趣的:(Keepalived、varnish)