Docker 封装高并发 OpenResty (Nginx Lua)

1. 准备

Docker 中有 openresty 的镜像, 我开始用这个镜像安装 lua, lua 安装成功, 但是 加载不上 resty.mysql 模块. 若想加载需要更新 ubuntu 资料库, 由于网络总是断, 所以换了一个思路. 用 Centos 镜像装一个.

这个过程比较多.

1. Docker 运行 centos 镜像.
2. 安装 Lua.
3. 安装 luarocks.
4. 安装 cjson 和 RestyMysql 模块.
5. 安装 openresty.
6. 打开防火墙.
7. 编译 lua 文件, 进行执行.

2. Docker 运行 centos 镜像

docker pull centos
docker run -d -it --net=host --name=centos centos

3. 安装 Lua

进入 Centos 镜像

docker exec -it centos /bin/bash

安装 Lua

curl -R -O http://www.lua.org/ftp/lua-5.1.tar.gz
tar zxf lua-5.1.tar.gz
cd lua-5.1
yum install -y gcc make
yum install -y readline-devel libtermcap-devel ncurses-devel libevent-devel
make linux
make install

运行 lua 命令, 进入控制台便运行成功.

防止后面的操作出现问题. 先打包镜像, 后续类似.

docker commit -a 'ykenan centos_lua' centos centos_lua

4. 安装 luarocks

curl -R -O https://luarocks.github.io/luarocks/releases/luarocks-3.3.1.tar.gz
tar -zxvf luarocks-3.3.1.tar.gz
cd luarocks-3.3.1
yum install unzip
./configure
make
make install

5. 安装 cjson 和 RestyMysql 模块

luarocks install lua-cjson
luarocks install lua-resty-mysql

若网络慢, 可以去 https://luarocks.org/modules 链接搜索下载下来再安装.

6. 安装 openresty

yum -y install readline-devel pcre-devel openssl-devel gcc pcre
wget https://openresty.org/download/ngx_openresty-1.9.7.1.tar.gz
tar xzvf ngx_openresty-1.9.7.1.tar.gz
cd ngx_openresty-1.9.7.1/
./configure --prefix=/opt/openresty --with-luajit --without-http_redis2_module --with-http_iconv_module
make
make install

yum install yum-utils
yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo

Nginx 中 nginx.conf 文件再 /opt/openresty/nginx/conf/ 文件夹下.

cat /opt/openresty/nginx/conf/nginx.conf

7. 打开防火墙

这步骤防止后面, restymysql 链接 MySQL 出现的报错.

yum install firewalld

systemctl start firewalld

打开用到的所有端口号. 比少了强.

1. 80 Nginx
2. 3306 MySQL
3. 6002 主机映射 Docker 80 端口
4. 6379 Redis

firewall-cmd --zone=public --add-port 80/tcp --permanent;
firewall-cmd --zone=public --add-port 3306/tcp --permanent;
firewall-cmd --zone=public --add-port 6002/tcp --permanent;
firewall-cmd --zone=public --add-port 6379/tcp --permanent;
firewall-cmd --reload;

查看防火墙的开放端口号

firewall-cmd --zone=public --list-all;

这里面会遇见 systemctl 运行的报错
https://blog.csdn.net/YKenan/article/details/107078114

先将容器打成镜像, 之后再用可以运行 systemctl 的方式运行该镜像即可.

8. 编译 lua 文件, 进行执行

防火墙打开之后重新打成镜像并运行.

docker run -d -it -p 6002:80 --privileged=true -v /home/docker/openresty/nginx:/opt/openresty/nginx/conf -v /home/docker/openresty/data:/root/data --name openresty centos_openresty /usr/sbin/init

1. /home/docker/openresty/nginx: Nginx 下的配置文件夹. (含有 nginx.conf)
2. /home/docker/openresty/data: 运行 lua 的文件夹. (含有 update_ad.lua)

docker exec -it openresty /bin/bash

yum install vim
vim ~/.bashrc
# 添加的内容-start
alias nginx='/opt/openresty/nginx/sbin/nginx'
# 添加的内容-end 保存推出
source ~/.bashrc

nginx -t
nginx -s reload

浏览器访问

nginx.conf 文件内容

user root;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
error_log  logs/error.log  info;

pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

	#定义缓存模块
	lua_shared_dict dis_cache 128m;

    server {
        listen       80;
        server_name  192.168.19.129;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }

		location /update_ad {
		    content_by_lua_file /root/data/update_ad.lua;
		}
		
		location /read_ad {
		    content_by_lua_file /root/data/read_ad.lua;
		}
		
        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }


    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}


    # HTTPS server
    #
    #server {
    #    listen       443 ssl;
    #    server_name  localhost;

    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;

    #    ssl_session_cache    shared:SSL:1m;
    #    ssl_session_timeout  5m;

    #    ssl_ciphers  HIGH:!aNULL:!MD5;
    #    ssl_prefer_server_ciphers  on;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

}

update_ad.lua 文件
restymysql 的使用 https://github.com/openresty/lua-resty-mysql

--ngx.header.content_type="text/plain;charset=utf8"
ngx.header.content_type="application/json;charset=utf8"
local cjson = require("cjson")
local mysql = require("resty.mysql")
local uri_args = ngx.req.get_uri_args()
local id = uri_args["id"]

local db, err = mysql:new()
if not db then
	ngx.say('{"failed to instantiate mysql":"'..err..'"}')
	return
end

db:set_timeout(3000)

local props = {
	host = "192.168.19.129",
	port = 3306,
	database = "commerce_ad",
	user = "root",
	password = "MySQL_6468",
	charset = "utf8"
}

local ok, err, errcode, sqlstate = db:connect(props)

if not ok then
	ngx.say('{"err":"'..err..'", "errcode":"'..errcode..'", "sqlstate":"'..sqlstate..'"}')
	return
end

local select_sql = "select url, pic_pos from tb_ad where status = '1' and category_id = "..id.." order by sort_order"
res, err, errcode, sqlstate = db:query(select_sql)
if not res then
	ngx.say('{"err":"'..err..'", "errcode":"'..errcode..'", "sqlstate":"'..sqlstate..'"}')
	return
end
db:close()

local redis = require("resty.redis")
local red = redis:new()
red:set_timeout(2000)
local ip = "192.168.19.129"
local port = 6379
red:connect(ip, port)
red:set("content_"..id, cjson.encode(res))
red:close()

ngx.say('[{"MySQL":"'..select_sql..'", "flag":"true"}]')

运行结果

http://192.168.19.129:6002/update_ad?id=1

9. 进行缓存数据读取

read_ad.lua 用于读取缓存数据.

读取过程:

1. 读取 Nginx 缓存, 若没有读取 Redis 缓存.
2. 读取 Redis 缓若存在, 将数据缓存到 Nginx 中. 若不存在读取数据库.
3. 读取数据库后将数据缓存到 Redis 中.

添加 Nginx 缓存模块

#定义缓存模块
lua_shared_dict dis_cache 128m;

read_ad.lua 文件内容

--ngx.header.content_type="text/plain;charset=utf8"
ngx.header.content_type="application/json;charset=utf8"
local uri_args = ngx.req.get_uri_args();
local id = uri_args["id"];
--获取本地缓存 (加载 Nginx 缓存模块, 该模块需要自己定义)
local cache_ngx = ngx.shared.dis_cache;
--根据 ID 获取本地缓存数据
local content_cache = cache_ngx:get('content_cache_'..id);
--进行判断
if content_cache == "" or content_cache == nil then
	local redis = require("resty.redis");
	local red = redis:new();
	red:set_timeout(2000);
	local ip = "192.168.19.129";
	local port = 6379;
	red:connect(ip, port);
	local rescontent = red:get("content_"..id);
	--进行判断
	if ngx.null == rescontent then
		local cjson = require("cjson")
		local mysql = require("resty.mysql")
		local db, err = mysql:new()
		if not db then
			ngx.say('{"failed to instantiate mysql":"'..err..'"}')
			return
		end
		db:set_timeout(2000)
		local props = {
			host = "192.168.19.129",
			port = 3306,
			database = "commerce_ad",
			user = "root",
			password = "MySQL_6468",
			charset = "utf8"
		}
		local ok, err, errcode, sqlstate = db:connect(props)
		if not ok then
			ngx.say('{"err":"'..err..'", "errcode":"'..errcode..'", "sqlstate":"'..sqlstate..'"}')
			return
		end
		local select_sql = "select url, pic_pos from tb_ad where status = '1' and category_id = "..id.." order by sort_order"
		res, err, errcode, sqlstate = db:query(select_sql)
		if not res then
			ngx.say('{"err":"'..err..'", "errcode":"'..errcode..'", "sqlstate":"'..sqlstate..'"}')
			return
		end
		local responsejson = cjson.encode(res);
		--添加 Redis 缓存
		red:set("content_"..id, responsejson);
		ngx.say(responsejson);
		db:close();
	else
		--将 Redis 查询出来的缓存放在 Nginx 中
		cache_ngx:set('content_cache_'..id, rescontent, 10*60);
		ngx.say(rescontent);
	end
	red:close();
else
	ngx.say(content_cache);
end