Nginx做文件服务器 支持上传和下载

Abstract

把nginx作为一个简单的http服务器. 因为可能涉及到加载额外的module 所以我们会从源码开始编译.

可选步骤 移除已安装的nginx module

这个步骤是可选的 如果在后面按照源码安装nginx后 加载其他module有冲突时:类似于下面的错误:

nginx: [emerg] module "/usr/lib64/nginx/modules/ngx_http_image_filter_module.so" is not binary compatible in /usr/share/nginx/modules/mod-http-image-filter.conf:1

可以通过如下方式移除(注意保存原有nginx配置 一般在/etc/nginx/)

yum remove nginx-mod*  

从源码安装nginx 并加载上传模块

wget https://nginx.org/download/nginx-1.16.1.tar.gz
# 下载upload module
git clone https://github.com/vkholodkov/nginx-upload-module/
tar -xvzf nginx-1.16.1.tar.gz
cd nginx-1.16.1
./configure --with-compat --add-dynamic-module=../nginx-upload-module
make install

如果出现如下错误: ./configure: error: the HTTP rewrite module requires the PCRE library.
解决办法：
yum -y install pcre-devel openssl openssl-devel

然后nginx被安装在: /usr/local/nginx/sbin 配置文件: /usr/local/nginx/conf

配置支持下载支持

可选的配置用户名密码访问

基于basic认证:

yum install httpd-tools
# 新建一个配置文件
htpasswd -c /usr/local/nginx/conf/htpasswd testuser
   会需要输入密码
  

下载配置

user root;
worker_processes  1;

pid        logs/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /usr/local/nginx/conf/mime.types;
    default_type        application/octet-stream;

    server {
        client_max_body_size 500m;
        listen       8097 default_server;
        listen       [::]:8097 default_server;
        server_name  _;
        autoindex   on;
        

        location / {
            root /root/nginxShare;
            auth_basic "security download area";
            auth_basic_user_file "/usr/local/nginx/conf/htpasswd";
            autoindex on;
        }
    }

}

注意我是以root启动的 映射目录是/root/nginxShare

运行截图

/usr/local/nginx/sbin/nginx

如果出现403, 需要检查配置的映射目录权限

配置上传支持

user root;
worker_processes  1;

pid        logs/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
load_module /usr/local/nginx/modules/ngx_http_upload_module.so;
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /usr/local/nginx/conf/mime.types;
    default_type        application/octet-stream;

    server {
        client_max_body_size 500m;
        listen       8097 default_server;
        listen       [::]:8097 default_server;
        server_name  _;
        autoindex   on;
        

        location / {
            root /root/nginxShare;
            auth_basic "security download area";
            auth_basic_user_file "/usr/local/nginx/conf/htpasswd";
            autoindex on;
        }
        
        location /upload {
              
              upload_store /root/nginxShare;
              upload_store_access user:rw;
              upload_set_form_field $upload_field_name.name "$upload_file_name";
              upload_set_form_field $upload_field_name.content_type "$upload_content_type";
              upload_set_form_field $upload_field_name.path "$upload_tmp_path";
              upload_pass_form_field "^submit$|^description$";
              upload_pass @mock;
        }
        location @mock {
              return 200 "ok";
        }
        location @filerewriter {
              proxy_pass http://localhost:9997;
        }
    }

}

上传测试

curl --basic -u testuser:testpass -X POST -F file=@/Users/edward/Downloads/word.xlsx "http://192.168.42.100:8097/upload"
ok%                  

注意这里上传后的文件名并不是本地的文件名而是000000这样的

如何上传后的文件是正确的名字呢?

这个其实比较麻烦, 关于讨论在这里, 尝试了文中 这里 的办法加入一个SessionId 并没有成功.
所以我自己写了个python 代码来rename (应该是lua也可以 只是不会). 启动了一个简单的bottle实现的服务器:

pip install bottle

server.py:

from bottle import *


@post("/upload")
def postExample():
    oldsimplename = request.forms.get("file.name")
    newfullname = request.forms.get("file.path")
    dir = os.path.abspath(os.path.join(newfullname,os.path.pardir))
    os.rename(newfullname,  os.path.join(dir, oldsimplename)  )

    return "ok"

run(host='localhost', port=9997)

修改nginx配置中的upload_pass @mock; 为 upload_pass @filerewriter; 然后reload:

nginx -s reload
python server.py

重新运行curl命令上传,这样上传的文件就是对的了:

参考

1. 官方的upload_module配置
2. 配置用户名密码