nginx 总结

nginx Fundamentals: High Performance Servers

nginx 是什么就不必废话了,高性能web服务器。下面还是一步步介绍:

安装

查看当前系统的信息:cat /etc/lsb-release

可以使用 apt-get install nginx 就可以按照默认的设定安装nginx

对于centos系统,可能不可以使用yum install nginx安装,会报没有nginx的源,这时候,就需要运行:yum install epel-release 安装所需要的源。然后再使用yum install nginx安装。

查看nginx所在的目录:/etc/nginx

查看nginx的服务运行状态:service nginx status/start/stop/restart/reload

使用源代码编译安装:

  1. 可能系统需要先update,然后安装build-essential。
  2. 安装可能需要的依赖: libpcre3 libpcre3-dev libpcrecpp0v5 libssl-dev zlib1g-dev
    其中的pcre包是regex相关的,用于nginx的配置;ssl相关用于ssl或者https,zlib相关用于压缩静态资源。
  3. 在“build nginx from source”的网页上(wiki.nginx.org)打开compile-time option页面,就可以看到相关的命令行参数信息,我们使用如下所示的配置方式:
    ./configure
    --sbin-path=/usr/bin/nginx
    --conf-path=/etc/nginx/nginx.conf
    --error-log-path=/var/log/nginx/error.log
    --http-log-path=/var/log/nginx/access.log
    --with-debug
    --with-pcre
    --with-http_ssl_module

或者使用没有空行的版本:
./configure --sbin-path=/usr/bin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --with-debug --with-pcre --with-http_ssl_module

点击 modules,查看上面命令行最后一个参数的信息。

  1. configure完成后,运行make,再make install。

配置服务

如果我们直接输入service nginx start,可能系统不能识别

debian/ubuntu安装脚本
初始化脚本

将上面的github中的代码下载到/etc/init.d/目录下,命名为nginx,然后使用chmod给该文件加上可执行权限,最后,使用命令:
update-rc.d -f nginx defaults 就可以使用service查看nginx的状态了

但是,此时我们输入 service nginx status, 会报错: you don't have the permission to execute nginx. 这是因为我们没有按照默认的配置安装nginx,解决的办法就是查看上面的github页面,里面有写:nginx是使用/etc/default/nginx里面的配置来作为 覆盖 所有配置的默认选项。

echo "NGINX_CONF_FILE=/etc/nginx/nginx.conf" > /etc/default/nginx
echo "DAEMON=/usr/bin/nginx" >> /etc/default/nginx

这样,我们就可以使用service nginx status查看状态了。

配置nginx

先看配置文件;一共有俩术语: contexts 和 directives。
context:类似与创建一个scope,server/http之类的context
directives:键值对

创建一个虚拟主机(virtual host)

首先,创建一个静态网站放在/sites/bootstrap 目录下,然后修改/etc/nginx/nginx.conf,得到一下内容:

events {}

http {

    server {
        listen 80:
        server_name 106.187.89.216;
        root /sites/bootstrap;

    }
}

此时发现nginx 可能没有发送一个 mind type(?)想要验证我们的想法,可以使用一下命令:
curl -I http:///css/bootstrap.css
可以看到content-type是text/plain

如果查看一下图片,使用一下命令:
curl -I http:///tile.png
得到的content-type同样是text/plain,这就需要修改配置文件:

events {}

http {

    include mime.types

    server {
        listen 80:
        server_name 106.187.89.216;
        root /sites/bootstrap;

    }
}

如果使用命令 head /etc/nginx/mime.types,可以看到mime.types所支持的类型。

location blocks

如果访问了不存在的url,nginx可以按照如下所示的方式定义

events {}

http {

    include mime.types

    server {
        listen 80:
        server_name 106.187.89.216;
        root /sites/bootstrap;

        ## matches any prefix
        # - /greet*
        # - /greeting
        # - /greet/something
        location /greet {
            return 200 'hello from nginx locations';
        }

        # exact match
        location = /greet {
            return 200 * "hello from nginx exact match location block!"
        }

        # regex match - case sensitive
        location ~ /greet[0-9] {
            return 200 * "hello from nginx case sensitive regex match location block!"
        }

        # regex match - case insensitive
        location *~ /greet[0-9] {
            return 200 * "hello from nginx case insensitive regex match location block!"
        }

        # prefix preferential match
        # same as location /greet, but more important than regex match
        location ^~ /greet[0-9] {
            return 200 * "hello from nginx match with preference over regex match.. location block!"
        }
    }
}

注意匹配顺序:

1. =    exact match
2. ^~   preferential prefix
3. ~ & *~ regex match
4. no modifier prefix match no modifier

还有一个特性是动态加载文件:假设我们在/sites目录下面创建一个/download目录,里面放一些图片和文本文件。可以用以下配置文件:

events {}

http {

    include mime.types

    server {
        listen 80:
        server_name 106.187.89.216;
        root /sites/bootstrap;

        location /downloads {
            root /sites;
            try_files $uri = 404
        }
    }
}

非常好,现在我们也看到了nginx配置文件中的第一个变量。

logging

已经在配置的一开始就设置了将日志信息输出的位置,当然,也可以制定某些特定错误的位置:

events {}

http {

    include mime.types

    server {
        listen 80:
        server_name 106.187.89.216;
        root /sites/bootstrap;

        location /downloads {
            error_log /var/log/nginx/download.error.log debug;

            root /sites;
            try_files $uri = 404
        }
    }
}

或者,可以关闭access log或者 error log:

events {}

http {

    include mime.types

    server {
        listen 80:
        server_name 106.187.89.216;
        root /sites/bootstrap;

        location /downloads {
            error_log off;
            access_log off;

            root /sites;
            try_files $uri = 404
        }
    }
}

配置文件的定义:

# array type directive
access_log logs/access.log;
access_log logs/access_notice.log notice;

http {
    include mime.types;

    #stardard directive
    gzip on;

    server {
        listen 80;
        server_name localhost;

        access_log logs/host.access.log main;

        location / {
            root html;
            index index.html index.htm;
        }

        location /home {
            rewrite ^ /index.html;
        }

        location /downloads {

            # standard directive
            gzip off;

            $ array type directive
            access_log logs/access_downloads.log main;

            $ try files directive
            try_files $uri =404;
        }
    }
}

关于这个uri的详细内容,可以参考http://nginx.org/en/docs/varindex.html

上述的所有内容,基本上就是随便哪个编程语言中的scope的套路。

配置动态语言的后端

user www-data www-data;

events {}

http {

    include /etc/nginx/mime.types

    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress

        index index.php index.html

        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        # pass all .php files onto a php-fpm/php-fcgi server.
        location ~ \.php${
            include fastcgi_params;
            include fastcgi.conf;

            fastcgi_pass 127.0.0.1:9000;
        }
    }
}

对于wordpress,进入/etc/php5/fpm/pool.d/www.conf, 修改33行的listen ,将其值指向 127.0.0.1:9000,然后重启nginx,php5-fpm,,就可以在网页上看到想要的结果,按照网页的要求,一步步配置数据库。

配置nginx的workers和其他directives

user www-data www-data;

# how many worker threads to run;
# auto set it to the number of cpu cores available in the system and offers the best
# performance, don't set it higher than the number of cpu cores if changing this parameter
worker_processes auto;

# maximum file descriptors that can be opened per process this should be just > worker connections
worker_rlimit_nofile 1035;
pid /var/run/nginx.pid;

events {
    worker_connections 1024; #最大打开时数量的文件描述符
    multi_accept on; # 总是异步接受新的连接,或者off将设定为接受
    use epoll;
}

http {

    # basic settings
    charset utf-8;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay off;
    types_hash_max_size 2048;

    # enable open file cache
    open_file_cache             max=1000 inactive=20s;
    open_file_cache_valid       30s;
    open_file_cache_min_uses    2;
    open_file_cache_errors      on;

    # configure buffer size
    client_body_buffer_size 16k;
    client_header_buffer_size 1k;
    client_max_body_size 8m;
    large_client_header_buffers 2 1k;

    #configure timeout
    client_body_timeout 12;
    client_header_timeout 12;

    # sue a higher keepalive timeout to reduce the need for repeated handshakes
    keepalive_timeout 300;

    send_timeout 10;

    # hide nginx version information.
    server_token off;

    # mime types
    include /etc/nginx/mime.types

    # add extra mime types    
    types {
        application/x-httpd-php .html;
    }

    default type application/octet-stream;

    access_log off;
    # access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    # gzip settings
    gzip on;
    gzip_ disable "msie6"
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 2;
    gzip_min_length 256;
    gzip_buffers 4 16k;
    gzip_http_version 1.1;

    # turn on gzip for all content types that should benefit from it.
    gzip_types application/ecmascript;
    gzip_types application/javascript;
    gzip_types application/json;
    gzip_types application/pdf;
    gzip_types application/postscript;
    gzip_types application/x-javascript;
    gzip_types image/svg+xml;
    gzip_types text/css;
    gzip_types text/csv;
    gzip_types text/javascript;
    gzip_types text/plain;
    gzip_types text/xml;

    # enable fastcgi caching
    fastcgi_cache_path /tmp/nginx-cache levels=1:2 keys_zone=WORDPRESS:50m inactive=3d;
    fastcgi_cache_key "$request_method$host$request_uri";
    fastcgi_cache_use_stale error timeout invalid_header http_500;
    fastcgi_ignore_headers Cache-Control Expires Set_Cookie;

    #######################################################################################
    # upstream to abstract backend connections for php.
    #######################################################################################

    upstream php {
        server unix:/var/run/php-fpm.sock;
    }

    #
    # rewrite requests for http://youdomain.ext to http://yourdomain.ext
    #

    server {
        listen 80;
        server_name {{host_domain}};
        return 301 https://{{host_domain}}$request_uri
    }

    # https server

    server {
        listen 443 ssl spdy;
        server_name {{ host_domain }};
        index index.php index.html index.htm;

        ssl_certificate                 {{ directory }}{{ sslcertificatechainfile }};
        ssl_certificate_key             {{ directory }}{{ sslcertificatekeyfile }};
        ssl_prefer_server_ciphers On;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

        # optimize SSL by caching session parameters for 10 minutes. this cuts down on the number of expensive SSL handshakes. the handshake is the most CPU-intensive operation, and by default it is re-negotiated on every new/parallel connections by enabling a cache (of type "shared between all Nginx workers") we tell the client to re-use the already negotiated further optimization can be archieve  by raising keepalive_timeout, but that shouldn't be done unless you serve primary.

        ssl_session_cache   shared:SSL:10m;
        ssl_session_timeout 24h;
        ssl_ciphers         ECDHE-RSA-AES256-GCM-SHA384;ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256

        # SSL buffer size was added in 1.5.9
        ssl_buffer_size 1400;

        # enable OCSP stapling
        ssl_stapling on;
        ssl_stapling_verify on;
        resolver 8.8.8.8 8.8.4.4 valid=60s;
        resolver_timeout 5s;

        # prevent mobile providers modding site
        add_header "cache-Control" "no-transform";

        # the X-Frame-Options header indicates whether a browser should be allowed
        # to render a page within a frame or iframe.
        add_header X-Frame-Options SAMEORIGIN;

        # enable HSTS
        add_header Strict-Transport-Security "max-age=3153600;";

        # inform brwowser of SPDY availability
        add_header Alternative-Protocol 443:npn-spdy/3;

        # adjust connection keepalive for spdy clients:
        spdy_keepalive_timeout 300; # up from 180 secs default

        # enable spdy header compressing
        spdy_headers_comp 6;

    }
}

在linux环境下,可以使用 nproc 查看当前的cpu 核心的数量, 使用 lscpu 将展示更多关于当前系统cpu的信息。

注意,worker_processes 和 worker_connections 的乘法作为当前系统所能支持的最大数量的连接,但是考虑到每个浏览器可能同时简历多个连接,所以所能支持的数量是这个数值的一半甚至四分之一.

可以使用ulimit -n 这个命令查看 Linux系统里打开文件描述符的最大值

dynamic modules

在nginx的安装目录,运行 ./configure --help | grep dynamic 查看那些可以使用dynamic module配置的选项,现在我们选中:
./configure --with-http_image_filter_module=dynamic

直接运行,会报错,说是系统没有安装GD包,下面我们使用如下所示的命令安装:apt install libgd2-xpm-dev,然后没有任何错误的运行:./configure --with-http_image_filter_module=dynamic & make & make install。

然后切换到nginx 的配置文件所在的位置:/usr/local/nginx/conf/nginx.conf

worker_processes 1;
events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    server {

        listen 80;
        server_name localhost;

        location / {
            root html;
            index index.html index.htm;
        }

        location = /img/logo.png {
            image_filter resize 300 300;
        }
    }
}

以上为默认的nginx 的配置文件,但是加上了图片显示的配置信息,重新加载 nginx,会报错:unknown directive 'image_filter' 这是因为该模块没有被正确的加载。

此时,如果查看/usr/local/nginx/modules, 会发现里面有 ngx_http_image_filter_module.so 文件,因此,就需要对于配置文件作出一下的修改:

load_module "modules/ngx_http_image_filter_module.so"

worker_processes 1;
events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    server {

        listen 80;
        server_name localhost;

        location / {
            root html;
            index index.html index.htm;
        }

        location = /img/logo.png {
            image_filter resize 300 300;
        }
    }
}

重启nginx,这次没有报错,然后重新加载图片,可以看到图片的尺寸已经被修改为300*300.

nginx的 location字段的匹配优先级:

  1. exact
  2. preferential prefix
  3. regex
  4. prefix

一下是网上找到的一段文字,惭愧,学的时候没有关注到这个点:

location表达式类型

~ 表示执行一个正则匹配,区分大小写

~* 表示执行一个正则匹配,不区分大小写

^~ 表示普通字符匹配。使用前缀匹配。如果匹配成功,则不再匹配其他location。

= 进行普通字符精确匹配。也就是完全匹配。

@ 它定义一个命名的 location,使用在内部定向时,例如 error_page, try_files



location优先级说明

在nginx的location和配置中location的顺序没有太大关系。正location表达式的类型有关。相同类型的表达式,字符串长的会优先匹配。

以下是按优先级排列说明:



等号类型(=)的优先级最高。一旦匹配成功,则不再查找其他匹配项。

^~类型表达式。一旦匹配成功,则不再查找其他匹配项。

正则表达式类型(~ ~*)的优先级次之。如果有多个location的正则能匹配的话,则使用正则表达式最长的那个。

常规字符串匹配类型。按前缀匹配。

location优先级示例

配置项如下:



location = / {

    # 仅仅匹配请求 /

    [ configuration A ]

}

location / {

    # 匹配所有以 / 开头的请求。

    # 但是如果有更长的同类型的表达式,则选择更长的表达式。

    # 如果有正则表达式可以匹配,则优先匹配正则表达式。

    [ configuration B ]

}

location /documents/ {

    # 匹配所有以 /documents/ 开头的请求。

    # 但是如果有更长的同类型的表达式,则选择更长的表达式。

    # 如果有正则表达式可以匹配,则优先匹配正则表达式。

    [ configuration C ]

}

location ^~ /images/ {

    # 匹配所有以 /images/ 开头的表达式,如果匹配成功,则停止匹配查找。

    # 所以,即便有符合的正则表达式location,也不会被使用

    [ configuration D ]

}

location ~* \.(gif|jpg|jpeg)$ {

    # 匹配所有以 gif jpg jpeg结尾的请求。

    # 但是 以 /images/开头的请求,将使用 Configuration D

    [ configuration E ]

}

请求匹配示例



/ -> configuration A

/index.html -> configuration B

/documents/document.html -> configuration C

/images/1.gif -> configuration D

/documents/1.jpg -> configuration E

注意,以上的匹配和在配置文件中定义的顺序无关。

提问:那种类型的directive 可以在同一个上下文中使用多次?

Array directive like the logging directive can be applied multiple times within the same context to specify multiple values. For example you could specify 3 different access logs by using the access_log directive with 3 different values.

nginx可以和后端交互使用哪两种方法?TCP port和unix socket

Expires header

将那些不长变更的数据加上缓存:

user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress;

        index index.php index.html;

        location / {
            try_files $uri $uri/ /index.php?$args;
        }
        location ~* \.(css|js|jpg\png\gif)$ {# <- 主要查看这一段
            assess_log off;
            expires 1M; # 或者 30d;两者等价
            add_header Pragma public;
            add_header Cache-Control public;
            add_header Vary Accept-Encoding;
        }
        location ~ \.php$ {

            include fastcgi_params;
            include fastcgi.conf;
            fastcgi_pass 127.0.0.1:9000;     
        }
    }
}

重启之后,可以查看到相关扩展名的文件,其已经被cached,而且也可以看到expired date的设定。

Gzip

我们想要查看某个文件是否支持GZip的方式,可以使用如下所示的命令查看:

curl -I -H 'Accept-Encoding: gzip, deflate' http:///wp-content/theme/twentyfifteen/style.css

似乎,没有关于gzip的任何信息。下面重新编辑configure 文件

user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress;

        index index.php index.html;

        # Gzip configuration
        gzip on;
        gzip_min_length 100;
        gzip_comp_level 3;

        gzip_types text/plain
        gzip_types text/css
        gzip_types text/javascript

        gzip_disable "msie6";


        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~* \.(css|js|jpg\png\gif)$ {# <- 主要查看这一段
            assess_log off;
            expires 1M; # 或者 30d;两者等价
            add_header Pragma public;
            add_header Cache-Control public;
            add_header Vary Accept-Encoding;
        }
        location ~ \.php$ {

            include fastcgi_params;
            include fastcgi.conf;
            fastcgi_pass 127.0.0.1:9000;     
        }
    }
}

然后查看配置文件是否编写正确,可以使用如下所示的命令:nginx -t

重启nginx,然后再次使用上面的curl命令,可以看到返回值里面有: Content-Encoding: gzip

fastcgi cache

运行bench test:ab -n 100 -c 10 http://46.101.19.11/
-n: 表示连接个数;100
-c: 表示10个并发连接

可以看到结果是需要等待五秒左右, requests per second 大概也就是几十左右;

user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    # FastCgi Cache <<<-----主要看这一段
    fastcgi_cache_path /tmp/nginx_cache levels=1:2 keys_zone=microcache:10m max_size=500m;
    fastcgi_cache_key  "$scheme$request_method$host$request_uri";
    add_header microcache-status $upstream_cache_status

    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress;

        index index.php index.html;

        # Gzip configuration
        gzip on;
        gzip_min_length 100;
        gzip_comp_level 3;

        gzip_types text/plain
        gzip_types text/css
        gzip_types text/javascript

        gzip_disable "msie6";



        # Default cache for everything << -- 还有这一段
        set $no_cache 0;

        # Bypass cache for POST requests
        if ($request_method = POST) { set $no_cache 1; }

        # Bypass cache for URL with query string
        if ($request_uri != "") { set $no_cache 1; }

        # Don't cache the following URLs
        if ($request_uri ~* "/wp-admin") { set $no_cache 1; }



        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~* \.(css|js|jpg\png\gif)$ {
            assess_log off;
            expires 1M; # 或者 30d;两者等价
            add_header Pragma public;
            add_header Cache-Control public;
            add_header Vary Accept-Encoding;
        }
        location ~ \.php$ {

            include fastcgi_params;
            include fastcgi.conf;

            fastcgi_cache microcache;  <<<- 加上这一段
            fastcgi_cache_valid 200 60m;

            fastcgi_cache_bypass $no_cache;
            fastcgi_no_cache $no_cache;

            fastcgi_pass 127.0.0.1:9000;     
        }
    }
}

现在加上了fastcgi的部分,再次运行ab命令,得到的requess per second大约是2800左右。提升效果显著啊。

现在再加上add_header那一段配置,然后再次重新启动nginx,可以用一下方式配置测试:
curl -I http://46.101.19.11/
可以看到microcache-status: HIT

如果将配置中的/tmp/nginx_cache 删除,再次运行以上的curl命令,得到的结果就是:microcache-status:MISS

然后设置$no_cache变量,使用的时候,可以看到microcache-status的bypass。

limiting

  1. concurrency 2. frequecy
user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    # FastCgi Cache <<<-----主要看这一段
    fastcgi_cache_path /tmp/nginx_cache levels=1:2 keys_zone=microcache:10m max_size=500m;
    fastcgi_cache_key  "$scheme$request_method$host$request_uri";
    add_header microcache-status $upstream_cache_status

    # limit concurrency
    # limit_conn_zone $server_name zone=per_vhost:5m;
    # limit_conn_zone $binary_remote_addr zone=per_ip:5m;
    limit_req_zone $binary_remote_addr zone=one_per_sec:5m rate=1r/s;

    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress;

        index index.php index.html;

        # Gzip configuration
        gzip on;
        gzip_min_length 100;
        gzip_comp_level 3;

        gzip_types text/plain
        gzip_types text/css
        gzip_types text/javascript

        gzip_disable "msie6";



        # Default cache for everything
        set $no_cache 0;

        # Bypass cache for POST requests
        if ($request_method = POST) { set $no_cache 1; }

        # Bypass cache for URL with query string
        if ($request_uri != "") { set $no_cache 1; }

        # Don't cache the following URLs
        if ($request_uri ~* "/wp-admin") { set $no_cache 1; }



        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~* \.(css|js|jpg\png\gif)$ {
            assess_log off;
            expires 1M; # 或者 30d;两者等价
            add_header Pragma public;
            add_header Cache-Control public;
            add_header Vary Accept-Encoding;

            # limit_conn per_ip 1;<---非常有用,限定每次只能有1个IP访问
            limit_req zone=one_per_sec burst=5;
        }


        location ~ \.php$ {

            include fastcgi_params;
            include fastcgi.conf;

            fastcgi_cache microcache;  
            fastcgi_cache_valid 200 60m;

            fastcgi_cache_bypass $no_cache;
            fastcgi_no_cache $no_cache;

            fastcgi_pass 127.0.0.1:9000;     
        }
    }
}

video streaming

需要加上--with_http_mp4_module,查看文档,可以看到一些相关的信息:

user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    # FastCgi Cache
    fastcgi_cache_path /tmp/nginx_cache levels=1:2 keys_zone=microcache:10m max_size=500m;
    fastcgi_cache_key  "$scheme$request_method$host$request_uri";
    add_header microcache-status $upstream_cache_status

    # limit concurrency
    # limit_conn_zone $server_name zone=per_vhost:5m;
    # limit_conn_zone $binary_remote_addr zone=per_ip:5m;
    limit_req_zone $binary_remote_addr zone=one_per_sec:5m rate=1r/s;

    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress;

        index index.php index.html;

        # Gzip configuration
        gzip on;
        gzip_min_length 100;
        gzip_comp_level 3;

        gzip_types text/plain
        gzip_types text/css
        gzip_types text/javascript

        gzip_disable "msie6";



        # Default cache for everything
        set $no_cache 0;

        # Bypass cache for POST requests
        if ($request_method = POST) { set $no_cache 1; }

        # Bypass cache for URL with query string
        if ($request_uri != "") { set $no_cache 1; }

        # Don't cache the following URLs
        if ($request_uri ~* "/wp-admin") { set $no_cache 1; }

        location ~ \.mp4$ { <<<---主要看这段
            root /sites/downloads/;
            mp4;
            mp4_buffer_size 4M;
            mp4_max_buffer_size 10M;
        }

        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~* \.(css|js|jpg\png\gif)$ {
            assess_log off;
            expires 1M; # 或者 30d;两者等价
            add_header Pragma public;
            add_header Cache-Control public;
            add_header Vary Accept-Encoding;

            # limit_conn per_ip 1;
            limit_req zone=one_per_sec burst=5;
        }


        location ~ \.php$ {

            include fastcgi_params;
            include fastcgi.conf;

            fastcgi_cache microcache;  
            fastcgi_cache_valid 200 60m;

            fastcgi_cache_bypass $no_cache;
            fastcgi_no_cache $no_cache;

            fastcgi_pass 127.0.0.1:9000;     
        }
    }
}

GeoIP

使用 --with-http_geoip_module
直接使用 ./configure --with-http_geoip_module 可能会有报错,提示缺少安装包,可以使用以下命令安装:apt install libgeoip-dev

然后再 make & make install 最后,运行命令nginx -V 查看安装和配置情况。

建立相关的目录:mkdir /etc/nginx/geoip

进入dev.maxmind.com 下载GeoLite2 free downloadable database,进入页面,下载GeoLite Country 和 GeoLite City,将这两个文件在在到上面建立的geoip目录下,然后使用gunzip命令解压缩。

下面修改配置文件

user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    # FastCgi Cache
    fastcgi_cache_path /tmp/nginx_cache levels=1:2 keys_zone=microcache:10m max_size=500m;
    fastcgi_cache_key  "$scheme$request_method$host$request_uri";
    add_header microcache-status $upstream_cache_status

    # limit concurrency
    # limit_conn_zone $server_name zone=per_vhost:5m;
    # limit_conn_zone $binary_remote_addr zone=per_ip:5m;
    # limit_req_zone $binary_remote_addr zone=one_per_sec:5m rate=1r/s;

    # GeoIP <<<<--- 主要看这段
    geoip_country /etc/nginx/geoip/GeoIP.dat
    geoip_city /etc/nginx/geoip/GeoLiteCity.dat


    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress;

        index index.php index.html;

        # Gzip configuration
        gzip on;
        gzip_min_length 100;
        gzip_comp_level 3;

        gzip_types text/plain
        gzip_types text/css
        gzip_types text/javascript

        gzip_disable "msie6";



        # Default cache for everything
        set $no_cache 0;

        # Bypass cache for POST requests
        if ($request_method = POST) { set $no_cache 1; }

        # Bypass cache for URL with query string
        if ($request_uri != "") { set $no_cache 1; }

        # Don't cache the following URLs
        if ($request_uri ~* "/wp-admin") { set $no_cache 1; }

        location /geo_country { <<<--主要看这段
            return 200 "visiting from $geoip_country_name"
        }

        location /geo_city { <<<--主要看这段
            return 200 "visiting from $geoip_city_name"
        }

        location ~ \.mp4$ {
            root /sites/downloads/;
            mp4;
            mp4_buffer_size 4M;
            mp4_max_buffer_size 10M;
        }

        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~* \.(css|js|jpg\png\gif)$ {
            assess_log off;
            expires 1M; # 或者 30d;两者等价
            add_header Pragma public;
            add_header Cache-Control public;
            add_header Vary Accept-Encoding;

            # limit_conn per_ip 1;
            limit_req zone=one_per_sec burst=5;
        }


        location ~ \.php$ {

            include fastcgi_params;
            include fastcgi.conf;

            fastcgi_cache microcache;  
            fastcgi_cache_valid 200 60m;

            fastcgi_cache_bypass $no_cache;
            fastcgi_no_cache $no_cache;

            fastcgi_pass 127.0.0.1:9000;     
        }
    }
}

HTTP2

  • binary protocol
  • header compression
  • persistent connections
  • multiplex streams
  • server push

简单的说:http2是基于Binary的协议(http1是基于文本的协议)header压缩,长连接。比如,在http1下,如果客户端请求一个index.html文件,后续会请求jquery.js和style.css三个文件,需要建立三个连接。

但是在HTTP2环境下,只需要建立一次请求,index.html,服务器端就会一次性返回三个文件。

开启HTTP2:使用 ./configure --with-http2_v2_module --with-http_ssl_module, 需要打开两个模块,是因为HTTP2默认需要使用加密。然后make&make install。

测试HTTP2是否配置成功,需要使用以下方法:

  • Online keycdn(ip not supported)<-没法用,仅仅支持域名
  • Chrome Extention
  • Latest version of curl
  • safari developer tools

查看 /usr/local/nginx/ssl/ 可以看到两个文件:nginx.crt, nginx.key

修改配置文件:/usr/local/nginx/conf/nginx.conf

user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    sendfile        on;
    keepalive_timeout 60;

    # HTTPS server
    server  {
        listen      443 ssl http2; <---主要看这里
        server_name localhost;

        ssl_certificate         /usr/local/nginx/ssl/nginx.crt; <---主要看这里
        ssl_certificate_key     /usr/local/nginx/ssl/nginx.key;

        ssl_session_cache       shared:SSL:1m;
        ssl_session_timeout     5m;

        ssl_ciphers             HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;

        location / {
            root    html;
            index   index.html index.htm
        }

    }

    }
}

其实,就是安装好相应的模块,然后配置好配置文件就可以了。

问题:尝试的时候,总是看到浏览器中https上有一个横线,似乎https并没启用,略奇怪。。还需要继续挖掘。

SSL

创建目录:mkdir /etc/nginx/ssl
运行命令: sudo openssl req -x509 -nodes -days 365 -newky rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
这个命令将生成key 文件和certificate 文件。更详细内容,可以搜索:how to create an ssl certificate on nginx for ubuntu 14.04。

然后配置:
ssl_certificate /usr/local/nginx/ssl/nginx.crt; <---主要看这里
ssl_certificate_key /usr/local/nginx/ssl/nginx.key;

参考:
http://nginx.org/en/docs/http/ngx_http_ssl_module.html
https://www.digitalocean.com/community/tutorials/how-to-create-an-ssl-certificate-on-nginx-for-ubuntu-14-04

basic auth

首先安装相应的包:apt install apache2-utils
创建密码:sudo htpasswd -c /etc/nginx/.htpassw ,然后输入密码

user www-data www-data;

worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include         mime.types;
    default_type    application/octet-stream;

    # FastCgi Cache
    fastcgi_cache_path /tmp/nginx_cache levels=1:2 keys_zone=microcache:10m max_size=500m;
    fastcgi_cache_key  "$scheme$request_method$host$request_uri";
    add_header microcache-status $upstream_cache_status

    # limit concurrency
    # limit_conn_zone $server_name zone=per_vhost:5m;
    # limit_conn_zone $binary_remote_addr zone=per_ip:5m;
    limit_req_zone $binary_remote_addr zone=one_per_sec:5m rate=1r/s;

    server {

        listen 80;
        server_name localhost;
        root /sites/wordpress;

        index index.php index.html;

        # Gzip configuration
        gzip on;
        gzip_min_length 100;
        gzip_comp_level 3;

        gzip_types text/plain
        gzip_types text/css
        gzip_types text/javascript

        gzip_disable "msie6";



        # Default cache for everything
        set $no_cache 0;

        # Bypass cache for POST requests
        if ($request_method = POST) { set $no_cache 1; }

        # Bypass cache for URL with query string
        if ($request_uri != "") { set $no_cache 1; }

        # Don't cache the following URLs
        if ($request_uri ~* "/wp-admin") { set $no_cache 1; }

        location ~ \.mp4$ {
            root /sites/downloads/;

            auto_basic "Restricted Content"; #<<<---主要看这段
            auto_basic_user_file /etc/nginx/.htpassw;

            mp4;
            mp4_buffer_size 4M;
            mp4_max_buffer_size 10M;
        }

        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~* \.(css|js|jpg\png\gif)$ {
            assess_log off;
            expires 1M; # 或者 30d;两者等价
            add_header Pragma public;
            add_header Cache-Control public;
            add_header Vary Accept-Encoding;

            # limit_conn per_ip 1;
            limit_req zone=one_per_sec burst=5;
        }


        location ~ \.php$ {

            include fastcgi_params;
            include fastcgi.conf;

            fastcgi_cache microcache;  
            fastcgi_cache_valid 200 60m;

            fastcgi_cache_bypass $no_cache;
            fastcgi_no_cache $no_cache;

            fastcgi_pass 127.0.0.1:9000;     
        }
    }
}

看mp4的部分,重启nginx,就可以看到如果想要访问mp4的资源,必须输入账号密码。

Hardening nginx

首先,移除不用的nginx模块,查看以下那些可以删除:
./configure --help | grep without

可以看到一个模块叫做 --without-http_autoindex_module

关闭显示 nginx的版本信息

在配置文件的http 中加上 server_tokens off

set buffer size

还是在配置文件的http中加上:

configure buffer sizes

client_body_buffer_size 16k;
client_header_buffer_size 1k;
client_max_body_size 8m;
large_client_header_buffers 2 1k;
关于更详细信息,还是查看相关的文档。

block user agents

这样可以禁止某些agent的访问,达到部分的反爬虫的效果:
if ($http_user_agent ~* badbot) {
return 403;
}

重启nginx,然后再使用curl -I -A "BadBot" 。可以得到的403的返回值
if ($http_referer ~* badbot) {
return 403;
}

configure X-Frame-Options

# the X-Frame-Options header indicates whether a browser should be allowed to
# rnder a page within a frame or iframe
add_header X-Frame-Options SAMEORIGIN;

更多信息,参考 http://modsecurity.org

如果想要删除默认安装的库,可以这么实现:
In order to compile and install Nginx without certain default modules, we have to pass the --without flag for those modules during the configure step. This can only be done when installing Nginx from source.

nginx的反向代理

可以用nginx启动一个web:localhost:8888,然后再用Flask启动一个webapp:localhost:8000,但是,我们怎么通过nginx访问使用flask建立的webapp呢?可以通过修改nginx配置文件的方式:

events {}

http {
    server {
        listen 8888;

        location / {
            return 200 "hello from nginx\n";
        }
        location /flask {
            proxy_pass 'http://localhost:8000/flask';
        }
    }
}

reload nginx server 通过:nginx -s reload

https://www.nginx.com/resources/admin-guide/reverse-proxy/
http://nginx.org/en/docs/http/ngx_http_proxy_module.html

load balance 负载均衡

简单的说,就是将nginx作为一个路由,后端有N个相同的消费者,nginx一次将需要处理的任务交付给这些消费者,如果某一个消费者宕机,剩余的消费者会自动接管任务的消费,如果宕机的消费者恢复,还可以继续从nginx那边接受任务。

此时,需要创建一个文件名为:load-balance.conf:

events {}

http {

    upstream flask_server {
        server localhost:8001;
        server localhost:8002;
        server localhost:8003;
    }

    server {
        listen 8888;
        location / {
            proxy_pass http://flask_server;
        }
    }
}

测试的时候,需要开启三个终端,开启三个flask 实例,然后再打开一个终端,重启nginx,然后运行命令:while sleep 1; do curl http://localhost:8000; done

然后随机关闭三个终端里面的flask实例,再打开。

结果:三个终端的端口依次消费,如果终止某一个终端,则消费端就不显示已经终止的终端。如果再将这个终端开启,则继续恢复显示这个终端。

以下,将查看sticky session(IP hashing)

events {}

http {

    upstream flask_server {
        ip_hash;                #<---关注这一行
        server localhost:8001;
        server localhost:8002;
        server localhost:8003;
    }

    server {
        listen 8888;
        location / {
            proxy_pass http://flask_server;
        }
    }
}

测试的时候,和上面例子一样,开启三个终端,还是需要执行这个脚本:while sleep 1; do curl http://localhost:8000; done
可以看到和上面的例子结果不一样,每次都是通过第一个终端消费,如终端1,如果终止这个终端1,则继续用终端2 消费,如果终端1 恢复,则恢复使用终端1 消费。

events {}

http {

    upstream flask_server {
        least_conn;                #<---关注这一行
        server localhost:8001;
        server localhost:8002;
        server localhost:8003;
    }

    server {
        listen 8888;
        location / {
            proxy_pass http://flask_server;
        }
    }
}

假设某一个请求特别耗时,这就需要依照服务器的负载分布任务而不是按照顺序分布任务。

参考文档:
http://nginx.org/en/docs/http/load_balancing.html
https://www.nginx.com/resources/admin-guide/load-balancer/
http://nginx.org/en/docs/http/ngx_http_upstream_module.html

有用的在线资源

http://nginx.org/en/docs/
https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
http://codex.wordpress.org/Nginx
https://github.com/facmbus/nginx-resources

1

你可能感兴趣的:(nginx 总结)