Haproxy+keepalived部署配置

[[TOC]]

haproxy配置

全局配置

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        #stats socket /run/haproxy/admin.sock mode 660 level admin
        stats timeout 30s
        user haproxy
        group haproxy
        daemon
        nbproc 1

defaults
        log     global
        timeout connect 5000
        timeout client  500000
        timeout server  500000

listen  admin_status                         #Frontend和Backend的组合体,监控组的名称，按需自定义名称 
         bind 0.0.0.0:8888                   #监听端口 
         mode http                           #http的7层模式 
         log 127.0.0.1 local3 err            #错误日志记录 
         stats refresh 5s                    #每隔5秒自动刷新监控页面 
         stats uri /stats                    #监控页面的url访问路径 
         stats realm wuhan united\ welcome   #监控页面的提示信息 
         stats auth admin:1qaz@WSX           #监控页面的用户和密码admin,可以设置多个用户名 
         stats hide-version                  #隐藏统计页面上的HAproxy版本信息  
         stats admin if TRUE  

控制节点负载均衡配置

提供唯一的apiserver地址：10.6.110.61:6443

listen kube-master
        bind 0.0.0.0:8443
        mode tcp
        option tcplog
        balance roundrobin
        server master1  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1
        server master2  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1
        server master3  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1

生产业务负载均衡配置

frontend http_frontend
   bind *:80
   acl is_http hdr_beg(host) *.uihcloud.cn
   redirect scheme https if !{ ssl_fc }
   mode http
   option httpclose
   option forwardfor
   reqadd X-Forwarded-Proto:\ https

frontend https_ingress
  bind *:443
  mode tcp
  default_backend https_web_server
backend https_web_server
  mode tcp
  balance roundrobin
  stick-table type ip size 200k expire 30m
  stick on src
  server s1  x.x.x.x:1234
  server s2  x.x.x.x:1234
  server s3  x.x.x.x:1234

传递真实客户端IP到后端配置

1.先ingress开启proxy_protocol协议，详细参照：
Haproxy+Ingress_nginx传递真实客户端IP到后端
2.haproxy配置修改，需要在转发server 后添加send-proxy
server s1 x.x.x.x:1234 send-proxy
server s2 x.x.x.x:1234 send-proxy
server s3 x.x.x.x:1234 send-proxy

完整配置文件：

主备一致

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        #stats socket /run/haproxy/admin.sock mode 660 level admin
        stats timeout 30s
        user haproxy
        group haproxy
        daemon
        nbproc 1

defaults
        log     global
        timeout connect 5000
        timeout client  500000
        timeout server  500000

listen  admin_status                         #Frontend和Backend的组合体,监控组的名称，按需自定义名称 
         bind 0.0.0.0:8888                   #监听端口 
         mode http                           #http的7层模式 
         log 127.0.0.1 local3 err            #错误日志记录 
         stats refresh 5s                    #每隔5秒自动刷新监控页面 
         stats uri /stats                    #监控页面的url访问路径 
         stats realm wuhan united\ welcome   #监控页面的提示信息 
         stats auth admin:1qaz@WSX           #监控页面的用户和密码admin,可以设置多个用户名 
         stats hide-version                  #隐藏统计页面上的HAproxy版本信息  
         stats admin if TRUE  

listen kube-master
        bind 0.0.0.0:8443
        mode tcp
        option tcplog
        balance roundrobin
        server master1  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1
        server master2  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1
        server master3  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1


listen prod-solar-logreciver
        bind 0.0.0.0:30216
        mode http
        timeout client  300000
        option  http-server-close
        option forwardfor
        server solar-reciver01  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1
        server solar-reciver02  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1
        server solar-reciver03  x.x.x.x:1234  check inter 10000 fall 2 rise 2 weight 1


#############################生产业务配置#################################
frontend http_frontend
   bind *:80
   acl is_http hdr_beg(host) *.uihcloud.cn
   redirect scheme https if !{ ssl_fc }
   mode http
   option httpclose
   option forwardfor
   reqadd X-Forwarded-Proto:\ https

frontend https_ingress
  bind *:443
  mode tcp
  default_backend https_web_server

backend https_web_server
  mode tcp
  balance roundrobin
  stick-table type ip size 200k expire 30m
  stick on src
  server s1  x.x.x.x:1234 send-proxy
  server s2  x.x.x.x:1234 send-proxy
  server s3  x.x.x.x:1234 send-proxy

keepaived 配置

两台keepalived实现高可用，提供唯一访问地址：10.6.110.61

主master配置

global_defs {
    router_id prod-backup
}

vrrp_instance prod-kube-master {
    state MASTER
    priority 110
    dont_track_primary
    interface ens192
    virtual_router_id 91
    advert_int 1
    authentication {
    auth_type PASS
    auth_pass 1qaz@WSX
    }
    virtual_ipaddress {
        x.x.x.x/24
    }
}

备backup配置

global_defs {
    router_id prod-backup
}

vrrp_instance prod-kube-master {
    state BACKUP
    priority 90
    dont_track_primary
    interface ens192
    virtual_router_id 91
    advert_int 1
    authentication {
    auth_type PASS
    auth_pass 1qaz@WSX
    }

    virtual_ipaddress {
        x.x.x.x/24
    }
}

将keepalived.service与haproxy.service加入开机启动中

systemctl enable haproxy.service
systemctl enable keepalived.service

systemctl is-enabled keepalived.service查看输出为：enabled 表示添加成功。