Ngxin详解之反向代理、负载均衡、缓存、URL重写及读写分离
大纲
一、环境准备
二、编译安装Nginx
三、Nginx反向代理
四、Nginx负载均衡
五、Nginx缓存功能
六、Nginx之URL重写
七、Nginx读写分离
一、环境准备
系统环境
CentOS5.8 x86_64
172.16.1.101 nginx
172.16.1.102 web1
172.16.1.103 web2
软件包
nginx-1.8.0.tar.gz
1、时间同步
[root@nginx ~]# ntpdate s2c.time.edu.cn 20 Jan 10:20:44 ntpdate[31442]: adjust time server 202.112.10.36 offset -0.014753 sec [root@web1 ~]# ntpdate s2c.time.edu.cn 20 Jan 10:12:53 ntpdate[31373]: adjust time server 202.112.10.36 offset -0.007873 sec [root@web2 ~]# ntpdate s2c.time.edu.cn 20 Jan 10:12:46 ntpdate[4132]: step time server 202.112.10.36 offset 1432097.703563 sec
2、关闭iptables和selinux
nginx [root@nginx ~]# service iptables stop [root@nginx ~]# chkconfig iptables off [root@nginx ~]# sed -r -i "s/^(SELINUX=).*/\1permissive/g" /etc/sysconfig/selinux [root@nginx ~]# setenforce 0 [root@nginx ~]# getenforce Permissive web1 [root@web1 ~]# service iptables stop [root@web1 ~]# chkconfig iptables off [root@web1 ~]# sed -r -i "s/^(SELINUX=).*/\1permissive/g" /etc/sysconfig/selinux [root@web1 ~]# setenforce 0 [root@web1 ~]# getenforce Permissive web2 [root@web2 ~]# service iptables stop [root@web2 ~]# chkconfig iptables off [root@web2 ~]# sed -r -i "s/^(SELINUX=).*/\1permissive/g" /etc/sysconfig/selinux [root@web2 ~]# setenforce 0 [root@web2 ~]# getenforce Permissive
3、下载所需的软件包
nginx [root@nginx ~]# wget http://nginx.org/download/nginx-1.8.0.tar.gz
二、编译安装Nginx
1、解决依赖关系
[root@nginx ~]# yum groupinstall -y "Development Tools" "Development Libraries" [root@nginx ~]# yum install openssl-devel pcre-devel
2、创建nginx用户和组
[root@nginx ~]# groupadd -r nginx [root@nginx ~]# useradd -r -g nginx nginx
3、编译安装
[root@nginx ~]# tar xf nginx-1.8.0.tar.gz [root@nginx ~]# cd nginx-1.8.0 [root@nginx nginx-1.8.0]# ls auto CHANGES CHANGES.ru conf configure contrib html LICENSE Makefile man objs README src [root@nginx nginx-1.8.0]# ./configure \ --prefix=/usr \ --sbin-path=/usr/sbin/nginx \ --conf-path=/etc/nginx/nginx.conf \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --pid-path=/var/run/nginx/nginx.pid \ --lock-path=/var/lock/nginx.lock \ --user=nginx \ --group=nginx \ --with-http_ssl_module \ --with-http_flv_module \ --with-http_stub_status_module \ --with-http_gzip_static_module \ --http-client-body-temp-path=/var/tmp/nginx/client/ \ --http-proxy-temp-path=/var/tmp/nginx/proxy/ \ --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ \ --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \ --http-scgi-temp-path=/var/tmp/nginx/scgi \ --with-pcre 显示我们自定义的配置信息和编译选项 Configuration summary + using system PCRE library + using system OpenSSL library + md5: using OpenSSL library + sha1: using OpenSSL library + using system zlib library nginx path prefix: "/usr" nginx binary file: "/usr/sbin/nginx" nginx configuration prefix: "/etc/nginx" nginx configuration file: "/etc/nginx/nginx.conf" nginx pid file: "/var/run/nginx/nginx.pid" nginx error log file: "/var/log/nginx/error.log" nginx http access log file: "/var/log/nginx/access.log" nginx http client request body temporary files: "/var/tmp/nginx/client/" nginx http proxy temporary files: "/var/tmp/nginx/proxy/" nginx http fastcgi temporary files: "/var/tmp/nginx/fcgi/" nginx http uwsgi temporary files: "/var/tmp/nginx/uwsgi" nginx http scgi temporary files: "/var/tmp/nginx/scgi" 编译安装 [root@nginx nginx-1.8.0]# make && make install 补充： Nginx可以使用Tmalloc(快速、多线程的malloc库及优秀性能分析工具)来加速内存分配 使用此功能需要事先安装gperftools，而后在编译nginx添加--with-google_perftools_module选项即可
4、为nginx提供SysV init脚本
[root@nginx nginx-1.8.0]# vim /etc/rc.d/init.d/nginx
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemon
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /etc/nginx/nginx.conf
# config: /etc/sysconfig/nginx
# pidfile: /var/run/nginx.pid
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
nginx="/usr/sbin/nginx"
prog=$(basename $nginx)
NGINX_CONF_FILE="/etc/nginx/nginx.conf"
[ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx
lockfile=/var/lock/subsys/nginx
make_dirs() {
# make required directories
user=`nginx -V 2>&1 | grep "configure arguments:" | sed 's/[^*]*--user=\([^ ]*\).*/\1/g' -`
options=`$nginx -V 2>&1 | grep 'configure arguments:'`
for opt in $options; do
if [ `echo $opt | grep '.*-temp-path'` ]; then
value=`echo $opt | cut -d "=" -f 2`
if [ ! -d "$value" ]; then
# echo "creating" $value
mkdir -p $value && chown -R $user $value
fi
fi
done
}
start() {
[ -x $nginx ] || exit 5
[ -f $NGINX_CONF_FILE ] || exit 6
make_dirs
echo -n $"Starting $prog: "
daemon $nginx -c $NGINX_CONF_FILE
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
killproc $prog -QUIT
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
configtest || return $?
stop
sleep 1
start
}
reload() {
configtest || return $?
echo -n $"Reloading $prog: "
killproc $nginx -HUP
RETVAL=$?
echo
}
force_reload() {
restart
}
configtest() {
$nginx -t -c $NGINX_CONF_FILE
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart|configtest)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac
添加执行权限，并加入到服务列表中
[root@nginx nginx-1.8.0]# chmod +x /etc/rc.d/init.d/nginx
[root@nginx nginx-1.8.0]# chkconfig --add nginx
[root@nginx nginx-1.8.0]# chkconfig nginx on
[root@nginx nginx-1.8.0]# chkconfig --list nginx
nginx 0:off 1:off 2:on 3:on 4:on 5:on 6:off
5、启动Nginx服务
[root@nginx nginx-1.8.0]# service nginx start Starting nginx: [ OK ] 查看80端口是否处于监听状态 [root@nginx nginx-1.8.0]# netstat -tnlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 2820/portmap tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 30414/nginx tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 25126/sshd tcp 0 0 0.0.0.0:922 0.0.0.0:* LISTEN 2860/rpc.statd tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 24814/sshd tcp 0 0 :::22 :::* LISTEN 25126/sshd tcp 0 0 ::1:6011 :::* LISTEN 24814/sshd
6、测试访问
三、Nginx反向代理
1、首先准备一个web服务器，这里以web1演示
首先安装web服务，这里使用yum安装httpd，以httpd作为web服务器 [root@web1 ~]# yum install -y httpd
2、配置并启动httpd服务，测试能正常访问
创建目录，配置页面文件 [root@web1 ~]# mkdir /var/www/html/bbs [root@web1 ~]# echo "<h1>Forum On Apache Server</h1>" > var/www/html/bbs/index.html 启动httpd服务 [root@web1 ~]# service httpd start Starting httpd: [ OK ] 测试自己能否访问，当前web1的ip地址就是172.16.1.102 [root@web1 ~]# curl http://172.16.1.102/bbs/ <h1>Forum On Apache Server</h1>
3、配置Nginx作为反向代理
这是切换到nginx主机，配置其作为反向代理服务器
[root@nginx ~]# hostname
nginx
编辑配置文件，加入下面三行至server段中
[root@nginx ~]# vim /etc/nginx/nginx.conf
location /forum {
proxy_pass http://172.16.1.102/bbs;
}
4、用浏览器测试访问
再来看web1上的访问日志 [root@web1 ~]# tail /var/log/httpd/access_log 172.16.1.101 - - [20/Jan/2016:13:57:21 +0800] "GET /bbs/ HTTP/1.0" 200 32 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36" 可以看到源ip是nginx反向代理并不是我们真正的客户端，但是有时候我们需要记录真正的client便于做日志分析 此时我们只需要添加proxy模块的proxy_set_header指令即可实现，此指令的语法格式如下 Syntax: proxy_set_header field value; Default: proxy_set_header Host $proxy_host; proxy_set_header Connection close; Context: http, server, location
5、配置Nginx反向代理和web服务器，使web服务器能够记录真实客户端ip地址
编辑配置文件，修改反向代理location那一段如下
[root@nginx ~]# vim /etc/nginx/nginx.conf
location /forum {
proxy_pass http://172.16.1.102/forum/;
proxy_set_header X-Real-IP $remote_addr; # 其实就是添加此行
}
而后重新载入Nginx代理服务器
[root@nginx nginx]# service nginx reload
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Reloading nginx: [ OK ]
再来用浏览器访问一下，可以看到仍然没有记录真实客户端的ip地址
[root@web1 ~]# tail /var/log/httpd/access_log
172.16.1.101 - - [20/Jan/2016:13:57:21 +0800] "GET /bbs/ HTTP/1.0" 200 32 "-" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"
172.16.1.101 - - [20/Jan/2016:13:58:03 +0800] "GET /bbs/ HTTP/1.0" 200 32 "-" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"
因为我们web服务器访问日志记录格式中第一项记录的请求客户端的地址，也就是我们的Nginx反向代理服务器地址
修改web服务器端的访问日志格式，修改之前先备份原有的combined模式并注释,将第一个%h修改为%{X-Real-IP}i即可
[root@web1 ~]# vim /etc/httpd/conf/httpd.conf
#LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Real-IP}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
然后重启httpd服务或是重新载入都可以
[root@web1 ~]# service httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]
再用浏览器访问一下然后看httpd服务器的访问日志
[root@web1 ~]# tail /var/log/httpd/access_log
172.16.1.101 - - [20/Jan/2016:13:57:21 +0800] "GET /bbs/ HTTP/1.0" 200 32 "-" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"
172.16.1.101 - - [20/Jan/2016:13:58:03 +0800] "GET /bbs/ HTTP/1.0" 200 32 "-" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"
172.16.1.100 - - [20/Jan/2016:14:09:28 +0800] "GET /bbs/ HTTP/1.0" 200 32 "-" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"
可以看到客户端真实ip已经记录，这个172.16.1.100就是我自己物理主机的地址
四、Nginx负载均衡
1、准备两台web服务器，配置好页面并进行本地测试。
web1在上面反向代理的示例中已经安装了httpd，这里不需要再安装，只是配置好主页并测试即可 [root@web1 ~]# echo "<h1>web1</h1>" > /var/www/html/index.html [root@web1 ~]# service httpd restart Stopping httpd: [ OK ] Starting httpd: [ OK ] [root@web1 ~]# curl localhost <h1>web1</h1> web2 [root@web2 ~]# yum install -y httpd [root@web2 ~]# echo "<h1>web2</h1>" > /var/www/html/index.html [root@web2 ~]# service httpd start Starting httpd: [ OK ] [root@web2 ~]# curl localhost <h1>web2</h1>
2、配置Nginx提供负载均衡功能
首先定义一个upstream组，注意，此upstream需定义在server段之外
[root@nginx nginx]# pwd
/etc/nginx
[root@nginx nginx]# vim nginx.conf
upstream webservers {
server 172.16.1.102 weight=1;
server 172.16.1.103 weight=1;
}
然后再修改location段为如下所示
location / {
proxy_pass http://webservers;
proxy_set_header X-Real-IP $remote_addr;
}
重新载入nginx
[root@nginx nginx]# service nginx reload
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Reloading nginx: [ OK ]
用浏览器访问一下
再刷新一下
可以看到，已经实现了简单的负载均衡效果
3、为Nginx增加上游服务器的健康状态监测功能
修改配置文件中upstream段为如下所示
[root@nginx nginx]# vim nginx.conf
upstream webservers {
server 172.16.1.102 weight=1 max_fails=2 fail_timeout=2s;
server 172.16.1.103 weight=1 max_fails=2 fail_timeout=2s;
}
然后重新载入nginx
[root@nginx nginx]# service nginx reload
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Reloading nginx: [ OK ]
用浏览器访问一下
再刷新一下
此时停止web1上的httpd服务，一直刷新，显示的一直都是web2
[root@web1 ~]# service httpd stop Stopping httpd: [ OK ]
再让web1上的httpd服务启动起来，可以看到，web1重新被加入进来了
[root@web1 ~]# service httpd start Starting httpd: [ OK ]
再刷新一下
4、为Nginx增加sorry_server
修改配置文件中upstream段为如下所示
[root@nginx nginx]# vim nginx.conf
upstream webservers {
server 172.16.1.102 weight=1 max_fails=2 fail_timeout=2s;
server 172.16.1.103 weight=1 max_fails=2 fail_timeout=2s;
server 127.0.0.1:8080 backup;
}
再在配置文件中定义一个server
server {
listen 8080;
server_name localhost;
root /var/www/mantainance;
index index.html;
}
接着创建维护页面目录及文件
[root@nginx nginx]# mkdir /var/www/mantainance
[root@nginx nginx]# echo "<h1>Mantainance Time</h1>" > /var/www/mantainance/index.html
然后重新载入nginx
[root@nginx nginx]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@nginx nginx]# service nginx reload
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Reloading nginx: [ OK ]
此时停掉web1和web2上的httpd服务
[root@web1 ~]# service httpd stop
Stopping httpd: [ OK ]
[root@node2 ~]# service httpd stop
Stopping httpd: [ OK ]
再使用浏览器访问一下
此时再上线web1 [root@web1 ~]# service httpd start Starting httpd: [ OK ] 浏览器刷新
再上线web2 [root@web2 ~]# service httpd start Starting httpd: [ OK ] 浏览器刷新
可以看到，当所有的web服务器都宕机时，Nginx自身会加入sorry_server
补充知识1
nginx支持三种调度算法：ip_hash(相当于LVS的sh)，round_robin(相当于LVS的rr和wrr)，least_conn(相当于LVS的lc)
要想使用ip_hash算法，直接在upstream段中定义ip_hash即可，但是用ip_hash时，不能定义sorry_server
补充知识2
统计80端口服务的每种连接状态数目
[root@nginx nginx]# netstat -tan | awk '/:80\>/{S[$NF]++}END{for(A in S) {print A,S[A]}}'