搭建FastDFS + Nginx（含配置文件参考）

心之所向，身之所往，己不由心，身又岂能由己，心之所向，一鹿向羊

v2-fe6333bb93367bab52d252f441d35488_r.jpg

目录

• 版本总览
• 一、FastDFS
  • 1.1 安装所需依赖包
  • 1.2 安装libfastcommon
  • 1.3 安装fastdfs
  • 1.4 命令行
  • 1.5 测试
• 二、Nginx
  • 2.1 安装nginx
  • 2.2 命令行
  • 2.3 测试

---

版本总览

• libfastcommon-1.0.42
• fastdfs-5.11
• fastdfs-nginx-module-1.20
• nginx-1.16.1

---

一、FastDFS

1.1 安装所需依赖包

yum install git gcc gcc-c++ make automake autoconf libtool pcre pcre-devel zlib zlib-devel openssl-devel -y

1.2 安装libfastcommon

• github选择需要安装的release版本
  https://github.com/happyfish100/libfastcommon/releases

• 下载安装包
  wget https://github.com/happyfish100/libfastcommon/archive/V1.0.42.tar.gz

• 解压
  tar -zxvf V1.0.42.tar.gz

• 安装

cd libfastcommon-1.0.42/
./make.sh
./make.sh install

• 可能报错：

./make.sh:行14: gcc: 未找到命令
./make.sh:行15: ./a.out: 没有那个文件或目录
cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O3 -c -o hash.o hash.c  
make: cc：命令未找到
make: *** [hash.o] 错误 127

• 解决方案：

根据报错信息显示，应该为未安装gcc导致，故先安装gcc
yum -y install gcc automake autoconf libtool make
安装完成后，重新执行编译命令 ./make.sh 即可

1.3 安装fastdfs

• github选择需要安装的release版本
  https://github.com/happyfish100/fastdfs/releases

• 下载安装包
  wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz

• 解压
  tar -zxvf V5.11.tar.gz

• 编译
  ./make.sh

• 安装
  ./make.sh install

可能报错：

致命错误 fastcommon/buffered_file_writer.h：没有那个文件或目录

解决方案：

降低版本重新编译（暂时这么做，没有深究）

• 拷贝配置文件，修改配置

cd /etc/fdfs/
cp client.conf.sample client.conf
cp storage.conf.sample storage.conf
cp tracker.conf.sample tracker.conf

• 修改client.conf

# the base path to store log files，日志以及数据存储位置
base_path=/home/work/fastdfs/fastdfs-data
 
# tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address tracker_server的ip端口
tracker_server=192.168.10.63:22122

• 修改trackerd.conf

# the base path to store data and log files
base_path=/home/work/fastdfs/fastdfs-data

• 修改storaged.conf

# the base path to store data and log files
base_path=/home/work/fastdfs/fastdfs-data
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
store_path0=/home/work/fastdfs/fastdfs-data
# tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
tracker_server=192.168.10.63:22122

1.4 命令行

• 启动

sudo service fdfs_trackerd start
sudo service fdfs_storaged start

• 重启

sudo /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
sudo /usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

• 终止

sudo /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop
sudo /usr/bin/fdfs_storaged /etc/fdfs/storage.conf stop

注：
要是使用终止命令或者使用 kill -9 fdfs进程id 来终结fdfs_storaged服务的，然后用上面的启动命令是启动不了的，因为log日志问题，会出现一些错误

1.5 测试

• 测试上传（命令格式，语句 + 配置文件 + 要上传的文件）
  fdfs_upload_file /etc/fdfs/client.conf client.conf
  成功后会返回如 group1/M00/00/00/wKgBOVvulymAC1_kAAAFspt4Ii458.conf 结构返回体

• 测试下载
  fdfs_download_file /etc/fdfs/client.conf 上述操作返回的那一串

• 测试删除
  fdfs_delete_file /etc/fdfs/client.conf 上述操作返回的那一串

---

二、Nginx

2.1 安装nginx

• 在官网nginx最新stable版本，官网：http://nginx.org/en/download.html

• 下载
  wget http://nginx.org/download/nginx-1.16.1.tar.gz

• 解压
  tar -zxvf nginx-1.16.1.tar.gz

• 安装依赖
  yum install pcre-devel zlib zlib-devel openssl openssl-devel

• 下载FastDFS nginx模块源代码并解压
  wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz -SO fastdfs-nginx-module.tar.gz

• 修改fdfs nginx模块src目录下config文件

if test -n "${ngx_module_link}"; then
    ngx_module_type=HTTP
    ngx_module_name=$ngx_addon_name
    ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon"#修改
    ngx_module_libs="-lfastcommon -lfdfsclient"
    ngx_module_srcs="$ngx_addon_dir/ngx_http_fastdfs_module.c"
    ngx_module_deps=
    CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"自己fdfs nginx模块src目录下的/mod_fastdfs.conf\"'"#修改
    . auto/module
else
    HTTP_MODULES="$HTTP_MODULES ngx_http_fastdfs_module"
    NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fastdfs_module.c"
    CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon"#修改
    CORE_LIBS="$CORE_LIBS -lfastcommon -lfdfsclient"
    CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"自己fdfs nginx模块src目录下的/mod_fastdfs.conf\"'"#修改
fi

• 修改fdfs nginx模块src目录下mod_fastdfs.conf文件

# the base path to store log files
base_path=同上述fastdfs中配置的路径
 
# FastDFS tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=192.168.10.63:22122
 
# if the url / uri including the group name
# set to false when uri like /M00/00/00/xxx
# set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx
# default value is false
url_have_group_name = true
 
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
# must same as storage.conf
store_path0=同上述fastdfs中配置的路径

• nginx目录下执行如下命令配置
  ./configure --add-module=你的fdfs nginx路径/fastdfs-nginx-module-1.20/src/

• nginx目录下编译安装（若不安装上述module，得先执行./configure）

make
make install

• 修改nginx配置，修改nginx安装目录下的conf/nginx.conf，增加自定义server，或者修改默认的都行

   server {
        listen 18000;
        server_name localhost;
 
        location ~/group[0-9]/{
            ngx_fastdfs_module;
        }
 
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

2.2 命令行

• 启动nginx

cd /usr/local/nginx/sbin/
再执行命令
./nginx 开启
./nginx -s stop 此方式相当于先查出nginx进程id再使用kill命令强制杀掉进程
./nginx -s quit 此方式停止步骤是待nginx进程处理任务完毕进行停止
./nginx -s reload 配置生效需要重启nginx，不用先停止nginx再启动，即可将配置信息在nginx中生效。

• 配置文件位置（注意，并不是安装目录下的conf，而在下述目录）
  /user/local/nginx/conf/

• 日志文件位置
  /usr/local/nginx/logs

• 可能报错：

缺少httpconf文件

• 解决方案

复制fastdfs安装文件夹中的conf文件夹下的 http.conf 和 mime.types 文件，至自己路径/fastdfs-nginx-module-1.20/src 下，在重新执行 添加模块、编译、安装 一次，在启动nginx即可

• 开机自启动

vim /etc/rc.local
新增
/usr/local/nginx/sbin/nginx
即可

2.3 测试

• nginx测试，使用文件上传返回的文件id测试
  wget http://127.0.0.1:18000/上面测试上传文件返回的那一串

• 集群配置

• 前提是先搭建两个以上的fdfs服务

• 每个服务stroage.conf增加两个tracker_server

# tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
tracker_server=192.168.1.57:22122
tracker_server=192.168.1.211:22122

• 每个服务的client.conf中配置两个tracker_server

# tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
tracker_server=192.168.1.57:22122
tracker_server=192.168.1.211:22122

• 同样nginx模块mod_fastdfs.conf

# FastDFS tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=192.168.1.57:22122
tracker_server=192.168.1.211:22122

• 集群测试
  配置ok后，重启每个服务的trackerd server和storaged server
  查看trackerd.log日志

[2018-11-19 14:14:56] INFO - file: tracker_relationship.c, line: 389, selecting leader...
[2018-11-19 14:14:58] INFO - file: tracker_service.c, line: 979, the tracker leader is 192.168.1.57:22122

• 查看storaged.log日志

[2018-11-19 14:14:58] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 192.168.1.57:22122, as a tracker client, my ip is 192.168.1.211
[2018-11-19 14:14:58] INFO - file: storage_sync.c, line: 2732, successfully connect to storage server 192.168.1.57:23000
[2018-11-19 14:15:28] INFO - file: tracker_client_thread.c, line: 1263, tracker server 192.168.1.57:22122, set tracker leader: 192.168.1.57:22122

• 上传文件后，在通过nginx下载文件，测试即可。

• nginx配置文件参考
  nginx.conf

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;
    
    # 声明别名，指向后台服务ip:port
    upstream ceson-api {
        server 127.0.0.1:9997;
    }

    # 监听21001端口
    server {
        listen       21001;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        #location / {
        #    root   html;
        #    index  index.html index.htm;
        #}
        
        # 匹配url中带 /ceson-api/ 的所有请求，匹配成功后进入
        location /ceson-api/ {
            # 将请求转发至 http://上面声明的别名/xxx/
            proxy_pass http://ceson-api/serever-api/;
            proxy_redirect off;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-Forwarded-Port $server_port;
            client_max_body_size 500m;
            client_body_buffer_size 128k;
            proxy_connect_timeout 90;
            proxy_send_timeout 90;
            proxy_read_timeout 90;
            proxy_buffer_size 4k;
            proxy_buffers 4 32k;
            proxy_busy_buffers_size 64k;
            proxy_temp_file_write_size 64k;
        }
        
        # 匹配url中带 /group1/M00 的所有请求，匹配成功后进入
        location /group1/M00{
            # 在此目录下找文件
            alias   /home/work/fastDFS/fastdfs-data;
        }
        
        # 匹配url中带 / 的所有请求，匹配成功后进入（也就是默认访问的主页门户网站之类）
        location / {
            root /home/work/static/ceson-web;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }
    
    server {
        listen       21002;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        #location / {
        #    root   html;
        #    index  index.html index.htm;
        #}
        
        location /ceson-api/ {
            proxy_pass http://ceson-api/server-api/;
            proxy_redirect off;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-Forwarded-Port $server_port;
            client_max_body_size 500m;
            client_body_buffer_size 128k;
            proxy_connect_timeout 90;
            proxy_send_timeout 90;
            proxy_read_timeout 90;
            proxy_buffer_size 4k;
            proxy_buffers 4 32k;
            proxy_busy_buffers_size 64k;
            proxy_temp_file_write_size 64k;
        }
        
        location / {
            root /home/work/static/menhu;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }

    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

    # HTTPS server
    #
    #server {
    #    listen       443 ssl;
    #    server_name  localhost;

    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;

    #    ssl_session_cache    shared:SSL:1m;
    #    ssl_session_timeout  5m;

    #    ssl_ciphers  HIGH:!aNULL:!MD5;
    #    ssl_prefer_server_ciphers  on;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

}