nginx负载均衡实现SSL, nginx代理tcp端口
重要信息
https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-load-balancing-with-ssl-termination
-
ssl证书放置在负载均衡的服务器上，后端服务器是使用80端口通信；
-
负载均衡用80端口，后端使用443是不行的；
-
负载和后端都使用443也是无必要的，因为服务器之间的通信应该是安全的，仅在负载均衡配置可以节约工作量；
-
一开始是使用一台后台服务器和一台负载衡服务简单实现代理，后来把后端服务器网卡变成2个IP，为新增的IP制作虚拟主机，其数据和目录配置跟原网卡的虚拟主机一样，模拟成两个服务器跑一个网站
-
另外这样可以看到访问时数据通过不同服务器提供
- 后端服务器
网络配置
[root@draft conf.d]# ifconfig
ens33: flags=4163 mtu 1500
inet 192.168.87.133 netmask 255.255.255.0 broadcast 192.168.87.255
inet6 fe80::d8da:c84:7947:a438 prefixlen 64 scopeid 0x20
ether 00:0c:29:bd:e9:03 txqueuelen 1000 (Ethernet)
RX packets 1370237 bytes 1512470028 (1.4 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 606601 bytes 202843219 (193.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:1: flags=4163 mtu 1500
inet 192.168.87.134 netmask 255.255.255.0 broadcast 192.168.87.255
ether 00:0c:29:bd:e9:03 txqueuelen 1000 (Ethernet)
虚拟主机1
server {
listen 80;
server_name 192.168.87.133;
root /data/wwwroot/bbs.tany.com/;
index index.html index.htm index.php;
location ~ \.php$ {
root /data/wwwroot/bbs.tany.com;
fastcgi_pass unix:/tmp/www.socket;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /data/wwwroot/bbs.tany.com$fastcgi_script_name;
include fastcgi_params;
}
access_log /data/logs/bbs.access.log main;
}
虚拟主机2:
server {
listen 80;
server_name 192.168.87.134;
root /data/wwwroot/bbs.tany.com/;
index index.html index.htm index.php;
location ~ \.php$ {
root /data/wwwroot/bbs.tany.com;
fastcgi_pass unix:/tmp/www.socket;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /data/wwwroot/bbs.tany.com$fastcgi_script_name;
include fastcgi_params;
}
access_log /data/logs/bbs.access.log main;
}
新建test.php页面在主机根目录里（用于测试）：
增加一个参数在日志格式里（由于是同一台机器同一个日志文件，也可以看到服务器跳转情况）：
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" "$server_addr" ';
- 负载均衡服务器：
ens33: flags=4163 mtu 1500
inet 192.168.87.141 netmask 255.255.255.0 broadcast 192.168.87.255
inet6 fe80::2db5:b70a:8f5d:a691 prefixlen 64 scopeid 0x20
ether 00:0c:29:b9:de:de txqueuelen 1000 (Ethernet)
RX packets 84007 bytes 50680307 (48.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 45290 bytes 15340130 (14.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
虚拟主机配置文件：
upstream qq_com
{
server 192.168.87.133;
server 192.168.87.134;
}
server
{ listen 443 ssl;
listen 80;
ssl_certificate /etc/nginx/ssl/bbs.tany.com/server.crt;
ssl_certificate_key /etc/nginx/ssl/bbs.tany.com/server.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
server_name bbs.tany.com;
location /
{
proxy_pass http://qq_com;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
access_log /data/logs/daili.log main;
}
- 测试结果：
以上配置还没有实现http自动转到https，80的数据和443的数据都会通过负载均衡去到后端服务器，443访问一些页面会有问题；
要实现自动跳转到443,需要增加域名重定向配置，下面增加一套实现自动跳转的配置，443访问页面问题还未解决；
设置hosts:
192.168.87.141 bbs.tany.com
$ curl --cacert server.cert https://bbs.tany.com/test.php
Host: bbs.tany.com
Remote Address: 192.168.87.141
X-Forwarded-For: 192.168.87.1
X-Forwarded-Proto:
Server Address: 192.168.87.133
Server Port: 80
$ curl --cacert server.cert https://bbs.tany.com/test.php
Host: bbs.tany.com
Remote Address: 192.168.87.141
X-Forwarded-For: 192.168.87.1
X-Forwarded-Proto:
Server Address: 192.168.87.134
Server Port: 80
日志情况：
192.168.87.141 - - [24/Oct/2019:00:37:48 +0800] "GET /test.php HTTP/1.0" 200 148 "-" "curl/7.54.0" "192.168.87.1" "192.168.87.133"
192.168.87.141 - - [24/Oct/2019:00:37:54 +0800] "GET /test.php HTTP/1.0" 200 148 "-" "curl/7.54.0" "192.168.87.1" "192.168.87.134"
192.168.87.141 - - [24/Oct/2019:00:41:02 +0800] "GET /test.php HTTP/1.0" 200 148 "-" "curl/7.54.0" "192.168.87.1" "192.168.87.133"
192.168.87.141 - - [24/Oct/2019:00:41:05 +0800] "GET /test.php HTTP/1.0" 200 148 "-" "curl/7.54.0" "192.168.87.1" "192.168.87.134"
另外一套自动跳转https的配置
upstream qq_com
{
server 192.168.87.133;
# server 192.168.87.134;
}
server
{ listen 443 ssl;
ssl_certificate /etc/nginx/ssl/bbs.tany.com/server.crt;
ssl_certificate_key /etc/nginx/ssl/bbs.tany.com/server.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
server_name bbs.tany.com;
location /
{
proxy_pass http://qq_com;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
access_log /data/logs/daili.log main;
}
重定向：
server {
listen 80;
server_name bbs.tany.com;
rewrite ^/(.*)$ https://${server_name}/$1 permanent;
}
后端服务器上就是一个论坛的虚拟主机
server {
listen 80;
server_name 192.168.87.133;
root /data/wwwroot/bbs.tany.com/;
index index.html index.htm index.php
location ~ \.php$ {
root /data/wwwroot/bbs.tany.com;
fastcgi_pass unix:/tmp/www.socket;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /data/wwwroot/bbs.tany.com$fastcgi_script_name;
include fastcgi_params;
}
access_log /data/logs/bbs.access.log main;
}
- 问题图片
一些类似模板的php页面都出现问题；
需要程序上修改修复；或后端也使用ssl，还没实验成功；
nginx代理tcp端口
这是区别于nginx虚拟主机（http)的一个用法；
在主配置文件/etc/nginx/etc/nginx.conf里要新增与http平衡的一个模块
stream {
log_format proxy '$remote_addr [$time_local] '
'$protocol $status $bytes_sent $bytes_received '
'$session_time "$upstream_addr" '
'"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';
include /etc/nginx/conf.d/tcp/*.conf; #新增tcp目录，stream的主配置文件要放在tcp里；
}
新建子配置文件tcp.conf
upstream stream_backend {
server 192.168.87.137:22; #端口应该不是80就是代理端口的用法，使用stream模块，这个子配置文件放到http模块里会报错；
}
server {
listen 1122;
proxy_pass stream_backend;
access_log /data/logs/tcp.log proxy;
}
重新加载nginx
即可测试服务器1122端口代理了内网137机器的22端品，即ssh;
在可连通服务器的第三台机器上测试
ssh -p 1122 [email protected]
日志信息，需要关闭服务才生成；
[root@draft conf.d]# tail -f /data/logs/tcp.log
192.168.87.1 [22/Oct/2019:18:09:45 +0800] TCP 200 3293 3337 174.476 "192.168.87.137:22" "3337" "3293" "0.001"
另外一个完善一点的配置文件
upstream 134 {
hash $remote_addr consistent;
server 192.168.1.134:22;
}
server {
listen 13422;
proxy_connect_timeout 10s;
proxy_timeout 300s;
proxy_pass 134;
}