haproxy 负载均衡&配置http+https负载集群实战
haproxy 负载均衡&配置http+https负载集群实战
haproxy负载均衡
一、简介
HAProxy是一种开源的负载均衡和代理服务器软件，可以实现高可用性和性能优化。它通常用于将传入的请求分发到多个后端服务器，以达到负载均衡的目的。
Haproxy 是一个使用C语言编写的自由及开放源代码软件，其提供高可用性、负载均衡，以及基于TCP和HTTP的应用程序代理。
HAProxy是一个免费的负载均衡软件，可以运行于大部分主流的Linux操作系统上（CentOS、Ubuntu、Debian、OpenSUSE、Fedora、麒麟、欧拉、UOS）。
HAProxy提供了L4(TCP)和L7(HTTP)两种负载均衡能力，具备丰富的功能。HAProxy具备媲美商用负载均衡器的性能和稳定性。
二、工作原理
HAProxy的工作原理是通过分析传入的请求，并使用配置的规则来决定如何处理请求。它可以根据多种策略（如轮询、最小连接数、源IP地址等）将请求分发到后端服务器。HAProxy还支持会话保持，它可以确保具有相同会话标识符的请求都被发送到同一个后端服务器，以确保用户的一致性体验。
三、工作流程
HAProxy的工作流程如下：
监听端口：HAProxy监听一个或多个端口，接收传入的请求。
请求分发：根据配置的负载均衡规则，HAProxy将请求分发到一个或多个后端服务器。
响应返回：后端服务器处理请求并将响应返回给HAProxy。
响应传输：HAProxy将后端服务器的响应传输给客户端。
Haproxy是一款可以供高可用性、负载均衡和基于TCP和HTTP应用的代理软件
- 适用于负载较大的站点
- 运行在硬件上可支持数以万计的并发连接请求
2、Haproxy的特性
可靠性和稳定性非常好，可以与硬件级的F5负载均衡设备相媲美
最高可以同时维护40000-50000个并发连接，单位时间内处理的最大请求数为20000个，最大处理能力可达10Git/s
支持多达8 种负载均衡算法，同时也支持会话保持
支持虚拟主机功能，从而实现web负载均衡更加灵活
支持连接拒绝、全透明代理等独特功能
拥有强大的ACL支持，用于访问控制等特性
环境说明：
| 虚拟机版本 | IP | 服务 |
|---|---|---|
| centos8（DR） | 192.168.136.139 | haproxy |
| centos8（RS1） | 192.168.136.140 | httpd |
| centos8 (RS2) | 192.168.136.142 | httpd |
所有的虚拟机关闭防火墙和selinux
[root@DR ~]# systemctl stop --now firewalld
[root@DR ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@DR ~]# setenforce 0
[root@RS1 ~]# systemctl stop --now firewalld
[root@RS1 ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@RS1 ~]# setenforce 0
[root@RS2 ~]# systemctl stop --now firewalld
[root@RS2 ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@RS2 ~]# setenforce 0
在DR上面部署haproxy
# 创建haproxy用户
[root@DR ~]# useradd -r -M -s /sbin/nologin haproxy
# 安装依赖包
[root@DR ~]# yum -y install make gcc pcre-devel bzip2-devel openssl-devel systemd-devel --allowerasing
# 下载haproxy官网的稳定版本的安装包
[root@DR ~]# wget https://www.haproxy.org/download/2.7/src/haproxy-2.7.10.tar.gz
# 解压
[root@DR ~]# tar -xf haproxy-2.7.10.tar.gz
[root@DR ~]# ll
total 4104
-rw-------. 1 root root 1246 Jul 27 09:41 anaconda-ks.cfg
drwxrwxr-x. 13 root root 4096 Aug 9 10:05 haproxy-2.7.10
-rw-r--r--. 1 root root 4191948 Aug 9 10:25 haproxy-2.7.10.tar.gz
[root@DR ~]# cd haproxy-2.7.10
# 编译
[root@DR haproxy-2.7.10]# make clean
[root@DR haproxy-2.7.10]# make -j $(nproc) TARGET=linux-glibc USE_OPENSSL=1 USE_ZLIB=1 USE_PCRE=1 USE_SYSTEMD=1
过程省略........
# 指定目录安装
[root@DR haproxy-2.7.10]# make install PREFIX=/usr/local/haproxy
# 设置环境变量
[root@DR haproxy-2.7.10]# ln -s /usr/local/haproxy/sbin/* /usr/sbin/
[root@DR haproxy-2.7.10]# which haproxy
/usr/sbin/haproxy
# 配置内核参数
[root@DR haproxy-2.7.10]# cd
[root@DR ~]# echo 'net.ipv4.ip_nonlocal_bind = 1' >> /etc/sysctl.conf
[root@DR ~]# echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
[root@DR ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.ip_forward = 1
# 提供配置文件
[root@DR ~]# mkdir /etc/haproxy
[root@DR ~]# cat > /etc/haproxy/haproxy.cfg <访问负载均衡器的http页面测试
访问DR的ip，会将请求转发给后端服务器，第一次分配给了RS1，多次刷新，分配给RS2
在RS1和RS2中部署httpd
[root@RS1 ~]# yum -y install httpd
[root@RS1 ~]# systemctl start httpd
[root@RS1 html]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 511 *:80 *:*
LISTEN 0 128 [::]:22 [::]:*
[root@RS1 html]# echo "RS1" > /var/www/html/index.html
[root@RS2 ~]# yum -y install httpd
[root@RS2 ~]# systemctl start httpd
[root@RS2 ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 511 *:80 *:*
LISTEN 0 128 [::]:22 [::]:*
[root@RS2 ~]# echo "RS2" >> /var/www/html/index.html
网页访问！
这里要把RS1或者RS2 端口号该变，避免冲突
[root@RS2 ~]# vim /etc/httpd/conf/httpd.conf
Listen 8080
# 重启以下httpd服务
RS1和RS2生成证书
# 在RS1
[root@RS1 ~]# mkdir -p /etc/pki/CA
[root@RS1 ~]# cd /etc/pki/CA/
[root@RS1 CA]# mkdir private
[root@RS1 CA]# (umask 077;openssl genrsa -out private/cakey.pem 2048)
Generating RSA private key, 2048 bit long modulus (2 primes)
............+++++
.............+++++
e is 65537 (0x010001)
[root@RS1 CA]# ls private/
cakey.pem
[root@RS1 CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 365
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:www.tq.com
Organizational Unit Name (eg, section) []:www.tq.com
Common Name (eg, your name or your server's hostname) []:www.tq.com
Email Address []:[email protected]
[root@RS1 CA]# ls
cacert.pem private
# 生成密钥
[root@RS1 CA]# ls private/
cakey.pem
[root@RS1 CA]# mkdir certs newcerts crl
[root@RS1 CA]# touch index.txt && echo 01 > serial
[root@RS1 CA]# cd /etc/httpd/ && mkdir ssl && cd ssl
[root@RS1 ssl]# (umask 077;openssl genrsa -out httpd.key 2048)
Generating RSA private key, 2048 bit long modulus (2 primes)
...................................................+++++
............................................................+++++
e is 65537 (0x010001)
# 生成证书签署请求
[root@RS1 ssl]# openssl req -new -key httpd.key -days 365 -out httpd.csrIgnoring -days; not generating a certificate
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:www.tq.com
Organizational Unit Name (eg, section) []:www.tq.com
Common Name (eg, your name or your server's hostname) []:www.tq.com
Email Address []:[email protected]
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
# CA签署它提交上来的证书
[root@RS1 ssl]# openssl ca -in httpd.csr -out httpd.crt -days 365
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 1 (0x1)
Validity
Not Before: Oct 11 16:07:17 2023 GMT
Not After : Oct 10 16:07:17 2024 GMT
Subject:
countryName = CN
stateOrProvinceName = HB
organizationName = www.tq.com
organizationalUnitName = www.tq.com
commonName = www.tq.com
emailAddress = [email protected]
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
53:45:17:71:FB:27:89:71:F7:A4:35:0D:9D:42:F6:58:BF:2E:97:94
X509v3 Authority Key Identifier:
keyid:CD:BF:E9:20:5C:76:D3:E0:77:53:6D:FD:D5:4E:EE:0A:0C:CA:C7:33
Certificate is to be certified until Oct 10 16:07:17 2024 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@RS1 ssl]#
# 安装证书服务
[root@RS1 ssl]# yum -y install httpd-devel mod_ssl
# 修改配置文件
[root@RS1 ssl]# vim /etc/httpd/conf.d/ssl.conf
[root@RS1 ssl]# grep -Ev '^$|^#' /etc/httpd/conf.d/ssl.conf
Listen 443 https
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout 300
SSLCryptoDevice builtin
DocumentRoot "/var/www/html/www.tq.com" #取消注释，修改自己域名
ServerName www.tq.com:443 # 这行
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn
SSLEngine on
SSLHonorCipherOrder on
SSLCipherSuite PROFILE=SYSTEM
SSLProxyCipherSuite PROFILE=SYSTEM
SSLCertificateFile /etc/httpd/ssl/httpd.crt # 修改对的路径
SSLCertificateKeyFile /etc/httpd/ssl/httpd.key # 这行也是
SSLOptions +StdEnvVars
SSLOptions +StdEnvVars
BrowserMatch "MSIE [2-5]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
CustomLog logs/ssl_request_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
[root@RS1 ssl]#
# 创建https的网页文件
[root@RS1 ssl]# mkdir -p /var/www/html/www.wanf.com
[root@RS1 ssl]# rm -rf /var/www/html/www.wanf.com
[root@RS1 ssl]# mkdir -p /var/www/html/www.tq.com
[root@RS1 ssl]# echo "this is RS1 https" > /var/www/html/www.tq.com/index.html
[root@RS1 ssl]# systemctl restart httpd
在RS2中生成证书
# scp 把RS1中的证书复制过来
[root@RS2 ~]# cd /etc/httpd/ && mkdir ssl && cd ssl
[root@RS2 ssl]# scp [email protected]:/etc/httpd/ssl/httpd.crt /etc/httpd/ssl/
The authenticity of host '192.168.136.140 (192.168.136.140)' can't be established.
ECDSA key fingerprint is SHA256:k8DoJlJ1tVvvBL4kfXDMXrbAW4iWqhw2fnatIewmqRo.
Are you sure you want to continue connecting (yes/no/[fingerprint])? y
Please type 'yes', 'no' or the fingerprint: yes
Warning: Permanently added '192.168.136.140' (ECDSA) to the list of known hosts.
[email protected]'s password:
httpd.crt 100% 4579 1.5MB/s 00:00
[root@RS2 ssl]# scp [email protected]:/etc/httpd/ssl/httpd.key /etc/httpd/ssl/
[email protected]'s password:
httpd.key 100% 1679 1.5MB/s 00:00
[root@RS2 ssl]# ll
total 12
-rw-r--r--. 1 root root 4579 Oct 11 19:18 httpd.crt
-rw-------. 1 root root 1679 Oct 11 19:19 httpd.key
# 安装证书服务
[root@RS2 ssl]# yum -y install httpd-devel mod_ssl
# 修改配置文件，和RS1一样
[root@RS2 ssl]# vim /etc/httpd/conf.d/ssl.conf
[root@RS2 ssl]# grep -Ev '^$|^#' /etc/httpd/conf.d/ssl.conf
Listen 443 https
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout 300
SSLCryptoDevice builtin
DocumentRoot "/var/www/html/www.tq.com"
ServerName www.tq.com:443
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn
SSLEngine on
SSLHonorCipherOrder on
SSLCipherSuite PROFILE=SYSTEM
SSLProxyCipherSuite PROFILE=SYSTEM
SSLCertificateFile /etc/httpd/ssl/httpd.crt
SSLCertificateKeyFile /etc/httpd/ssl/httpd.key
SSLOptions +StdEnvVars
SSLOptions +StdEnvVars
BrowserMatch "MSIE [2-5]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
CustomLog logs/ssl_request_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
[root@RS2 ssl]#
#创建https网页文件
[root@RS2 ssl]# mkdir -p /var/www/html/www.tq.com
[root@RS2 ssl]# echo "this is RS2 https" > /var/www/html/www.tq.com/index.html
# 重启服务
[root@RS2 ssl]# systemctl restart httpd
配置https 负载均衡规则 (DR主机上)
[root@DR ~]# vim /etc/haproxy/haproxy.cfg
[root@DR ~]# cat /etc/haproxy/haproxy.cfg
#--------------全局配置----------------
global
log 127.0.0.1 local0 info
#log loghost local0 info
maxconn 20480
#chroot /usr/local/haproxy
pidfile /var/run/haproxy.pid
#maxconn 4000
user haproxy
group haproxy
daemon
#---------------------------------------------------------------------
#common defaults that all the 'listen' and 'backend' sections will
#use if not designated in their block
#---------------------------------------------------------------------
defaults
mode tcp // 修改为tcp
log global
option dontlognull
option httpclose
option httplog
#option forwardfor
option redispatch
balance roundrobin
timeout connect 10s
timeout client 10s
timeout server 10s
timeout check 10s
maxconn 60000
retries 3
#--------------统计页面配置------------------
listen admin_stats
bind 0.0.0.0:8189
stats enable
mode http
log global
stats uri /haproxy_stats
stats realm Haproxy\ Statistics
stats auth admin:admin
#stats hide-version
stats admin if TRUE
stats refresh 30s
#---------------web设置-----------------------
listen webcluster
bind 0.0.0.0:443 //修改为443
mode tcp // 改为tcp
#option httpchk GET /index.html
log global
maxconn 3000
balance roundrobin
cookie SESSION_COOKIE insert indirect nocache
server web03 192.168.136.140:443 check inter 2000 fall 5 //修改
server web04 192.168.136.142:443 check inter 2000 fall 5 //修改
#server web01 192.168.80.102:8080 cookie web01 check inter 2000 fall 5
# 重启服务
[root@DR ~]# systemctl restart haproxy.service
# 查看端口
[root@DR ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 2048 0.0.0.0:443 0.0.0.0:*
LISTEN 0 2048 0.0.0.0:8189 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
[root@DR ~]#
访问负载均衡https的页面
刷新
IP地址后接上8189/haproxy_stats就可以访问
里面是haproxy的负载集群主机的状态
# 查看
[root@DR ~]# cat /etc/haproxy/haproxy.cfg
#--------------统计页面配置------------------
listen admin_stats
bind 0.0.0.0:8189 //端口信息
stats enable
mode http
log global
stats uri /haproxy_stats
stats realm Haproxy\ Statistics
stats auth admin:admin //用户信息
#stats hide-version
stats admin if TRUE
stats refresh 30s
#---------------web设置-----------------------
listen webcluster
