CentOS 单机搭建FastDFS文件系统

此例为在CentOS7.2 单机上搭建一个FastDFS 文件管理系统

FastDFS架构图

软件名称	版本	百度云盘存放名称
FastDFS	5.11	fastdfs-5.11.zip
FastDFS-Nginx-module	无	fastdfs-nginx-module-master.zip
LibFastCommon	1.0.36	libfastcommon-1.0.36.zip
nginx	1.10.3	nginx-1.10.3.tar.gz

安装所需文件均上传到百度云盘，位置：FastDFS百度云盘
安装清单如下：

软件名称	版本	百度云盘存放名称
FastDFS	5.11	fastdfs-5.11.zip
FastDFS-Nginx-module	无	fastdfs-nginx-module-master.zip
LibFastCommon	1.0.36	libfastcommon-1.0.36.zip
nginx	1.10.3	nginx-1.10.3.tar.gz

一、安装FastDFS

1.安装libfastcommon

先解压安装包到目录

[root@localhost fastDFS]# unzip libfastcommon-1.0.36.zip 

解压后目录如下：

[root@localhost fastdfs-5.11]# ll
[root@localhost libfastcommon-1.0.36]# ll
总用量 32
drwxr-xr-x. 2 root root  117 4月   5 2017 doc
-rw-r--r--. 1 root root 8005 4月   5 2017 HISTORY
-rw-r--r--. 1 root root  566 4月   5 2017 INSTALL
-rw-r--r--. 1 root root 1606 4月   5 2017 libfastcommon.spec
-rwxr-xr-x. 1 root root 3099 4月   5 2017 make.sh
drwxr-xr-x. 2 root root  191 4月   5 2017 php-fastcommon
-rw-r--r--. 1 root root 2763 4月   5 2017 README
drwxr-xr-x. 3 root root 4096 1月  17 11:21 src

安装C编译工具 gcc

[root@localhost fastdfs-5.11]# yum -y install gcc-c++

编译libfastcommon软件并安装

[root@localhost fastdfs-5.11]# ./make.sh  && ./make.sh install

为libcommon 创建软链接到/usr/local/lib目录下

[root@localhost fastdfs-5.11]# ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
[root@localhost fastdfs-5.11]# ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
[root@localhost fastdfs-5.11]# ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
[root@localhost fastdfs-5.11]# ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so

2.安装FastDFS

解压安装包

[root@localhost fastDFS]# unzip fastdfs-5.11.zip 

解压后目录如下：

总用量 128
drwxr-xr-x. 3 root root  4096 1月  17 11:25 client
drwxr-xr-x. 2 root root  4096 1月  17 11:25 common
drwxr-xr-x. 2 root root   146 6月   3 2017 conf
-rw-r--r--. 1 root root 35067 6月   3 2017 COPYING-3_0.txt
-rw-r--r--. 1 root root  3171 6月   3 2017 fastdfs.spec
-rw-r--r--. 1 root root 33100 6月   3 2017 HISTORY
drwxr-xr-x. 2 root root    48 6月   3 2017 init.d
-rw-r--r--. 1 root root  7755 6月   3 2017 INSTALL
-rwxr-xr-x. 1 root root  5548 6月   3 2017 make.sh
drwxr-xr-x. 2 root root  4096 6月   3 2017 php_client
-rw-r--r--. 1 root root  2380 6月   3 2017 README.md
-rwxr-xr-x. 1 root root  1768 6月   3 2017 restart.sh
-rwxr-xr-x. 1 root root  1680 6月   3 2017 stop.sh
drwxr-xr-x. 4 root root  4096 1月  17 11:25 storage
drwxr-xr-x. 2 root root  4096 6月   3 2017 test
drwxr-xr-x. 2 root root  4096 1月  17 11:25 tracker

进入解压目录并进行编译和安装

[root@localhost fastDFS]# cd fastdfs-5.11/
[root@localhost fastdfs-5.11]# ./make.sh  && ./make.sh install

安装成功后，FastDFS会安装在/etc/fdfs目录下：

[root@localhost fastdfs-5.11]# ll /etc/fdfs/
总用量 76
-rw-r--r--. 1 root root   316 1月  17 11:47 client.conf
-rw-r--r--. 1 root root  1461 1月  17 11:25 client.conf.sample
-rw-r--r--. 1 root root   955 1月  17 13:20 http.conf
-rw-r--r--. 1 root root 31172 1月  17 13:21 mime.types
-rw-r--r--. 1 root root  3716 1月  17 12:57 mod_fastdfs.conf
-rw-r--r--. 1 root root  1278 1月  17 11:40 storage.conf
-rw-r--r--. 1 root root  7927 1月  17 11:25 storage.conf.sample
-rw-r--r--. 1 root root   105 1月  17 11:25 storage_ids.conf.sample
-rw-r--r--. 1 root root  1356 1月  17 11:34 tracker.conf
-rw-r--r--. 1 root root  7389 1月  17 11:25 tracker.conf.sample

我们需要把这三个示例文件复制一份，去掉.sample

[root@localhost fdfs]# cp client.conf.sample client.conf
[root@localhost fdfs]# cp storage.conf.sample storage.conf
[root@localhost fdfs]# cp tracker.conf.sample tracker.conf

FastDFS安装结束

二、安装Tracker

1.创建tracker工作目录

此目录用于保存tracker 的data和log

[root@localhost fdfs]# mkdir /opt/fastdfs_tracker

2.配置tracker

配置 /etc/fdfs目录下tracker.conf
主要实现以下4个配置内容：

1.disabled=false 
2.port=22122 #默认端口号 
3.base_path=/opt/fastdfs_tracker #我刚刚创建的目录 
4.http.server_port=8080 #默认端口是8080
5.bind_addr= 0.0.0.0 监听地址

完整tracker.conf 文件信息如下：

disabled=false

bind_addr= 0.0.0.0

port=22122

connect_timeout=30

network_timeout=60

base_path=/opt/fastdfs_tracker

max_connections=512

accept_threads=1

work_threads=4

min_buff_size = 8KB

max_buff_size = 128KB

store_lookup=2

store_group=group2

store_server=0

store_path=0

download_server=0

reserved_storage_space = 10%

log_level=info

run_by_group=

run_by_user=

allow_hosts=*

sync_log_buff_interval = 10

check_active_interval = 120

thread_stack_size = 64KB

storage_ip_changed_auto_adjust = true

storage_sync_file_max_delay = 86400

storage_sync_file_max_time = 300

use_trunk_file = false 

slot_min_size = 256

slot_max_size = 16MB

trunk_file_size = 64MB

trunk_create_file_advance = false

trunk_create_file_time_base = 02:00

trunk_create_file_interval = 86400

trunk_create_file_space_threshold = 20G

trunk_init_check_occupying = false

trunk_init_reload_from_binlog = false

trunk_compress_binlog_min_interval = 0

use_storage_id = false

storage_ids_filename = storage_ids.conf

id_type_in_filename = ip

store_slave_file_use_link = false

rotate_error_log = false

error_log_rotate_time=00:00

rotate_error_log_size = 0

log_file_keep_days = 0

use_connection_pool = false

connection_pool_max_idle_time = 3600

http.server_port=8080

http.check_alive_interval=30

http.check_alive_type=tcp

http.check_alive_uri=/status.html

修改保存后创建软引用

[root@localhost fdfs]# ln -s /usr/bin/fdfs_storaged /usr/local/bin

4.启动tracker，并加入开机启动项

[root@localhost fdfs]# service fdfs_trackerd start

进行刚刚创建的tracker目录，发现目录中多了data和log两个目录

[root@localhost fdfs]# ll /opt/fastdfs_tracker/
总用量 0
drwxr-xr-x. 2 root root 178 1月  17 13:48 data
drwxr-xr-x. 2 root root  26 1月  17 11:35 logs

将tracker加入开机启动项

[root@localhost fdfs]# echo "service fdfs_trackerd start" |tee -a /etc/rc.d/rc.local

查看一下tracker的端口监听情况

[root@localhost fdfs]# netstat -unltp|grep fdfs
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      3088/fdfs_trackerd  

三、安装Storage

1.配置storage工作目录

由于storage还需要一个目录用来存储数据，因此多建了两个目录fastdfs_storage_data,fastdfs_storage

[root@localhost opt]# mkdir fastdfs_storage
[root@localhost opt]# mkdir fastdfs_storage_data
[root@localhost opt]# ll
总用量 0
drwxr-xr-x. 4 root root 30 1月  17 11:45 fastdfs_storage
drwxr-xr-x. 3 root root 18 1月  17 11:45 fastdfs_storage_data
drwxr-xr-x. 4 root root 30 1月  17 11:35 fastdfs_tracker

2.配置storage文件

修改 /etc/fdfs 目录下 storage.conf 文件
修改要点如下：

1.disabled=false 
2.group_name=group1 #组名，根据实际情况修改 
3.port=23000 #设置storage的端口号，默认是23000，同一个组的storage端口号必须一致 
4.base_path=/opt/fastdfs_storage #设置storage数据文件和日志目录 
5.store_path_count=1 #存储路径个数，需要和store_path个数匹配 
6.store_path0=/opt/fastdfs_storage_data #实际文件存储路径 
7.tracker_server=192.168.43.60:22122 #我CentOS7的ip地址 
8.http.server_port=8888 #设置 http 端口号

完整信息如下：

disabled=false
group_name=group1
bind_addr= 0.0.0.0
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/opt/fastdfs_storage
max_connections=256
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/opt/fastdfs_storage_data
subdir_count_per_path=256
tracker_server=192.168.43.60:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
log_file_keep_days = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=8888

修改保存后创建软引用

[root@localhost fdfs]#  ln -s /usr/bin/fdfs_storaged /usr/local/bin

3.启动Storage

[root@localhost fdfs]# service fdfs_storaged start

设置开机启动：

[root@localhost fdfs]#  echo "service fdfs_storaged start" |tee -a /etc/rc.d/rc.local

查看一下服务是否启动

[root@localhost fdfs]# netstat -unltp | grep fdfs 
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      3088/fdfs_trackerd  
tcp        0      0 0.0.0.0:23000           0.0.0.0:*               LISTEN      3139/fdfs_storaged  

4.校验整合

到这里，fastdfs的东西都已安装完成，最后我们还要确定一下，storage是否注册到了tracker中去。
查看命令：

[root@localhost fdfs]# /usr/bin/fdfs_monitor /etc/fdfs/storage.conf

成功后可以看到：
ip_addr = 192.168.43.60 (localhost.localdomain) ACTIVE

四、测试

1.配置客户端

修改 /etc/fdfs/目录下的client.conf 文件
修改要点为：

base_path=/opt/fastdfs_tracker #tracker服务器文件路径
tracker_server=192.168.43.60:22122 #tracker服务器IP地址和端口号
http.tracker_server_port=8080 # tracker 服务器的 http端口号，必须和tracker的设置对应起来

完整client.conf 文件信息如下：

connect_timeout=30
network_timeout=60
base_path=/opt/fastdfs_tracker
tracker_server=192.168.43.60:22122
log_level=info
use_connection_pool = false
connection_pool_max_idle_time = 3600
load_fdfs_parameters_from_tracker=false
use_storage_id = false
storage_ids_filename = storage_ids.conf
http.tracker_server_port=8080

模拟上传
从个人用户目录上传一个图片，进行测试

[root@localhost fdfs]# fdfs_upload_file  /etc/fdfs/client.conf /home/zhangyongliang/9408.jpg  #这后面放的是图片的位置

成功后会返回图片存储路径

[root@localhost fdfs]# fdfs_upload_file  /etc/fdfs/client.conf /home/zhangyongliang/9408.jpg 
group1/M00/00/00/wKgrPFpe9OqAWsHxAAH5yvc2jn8251.jpg

组名：group1
磁盘：M00
目录：00/00
文件名称：wKgrPFpe9OqAWsHxAAH5yvc2jn8251.jpg
定位上传的文件位置如下：

[root@localhost fdfs]# ll /opt/fastdfs_storage_data/data/00/00/
总用量 256
-rw-r--r--. 1 root root 129482 1月  17 15:02 wKgrPFpe9OqAWsHxAAH5yvc2jn8251.jpg
-rw-r--r--. 1 root root 129482 1月  17 11:53 wKgrPFpeyM2ATkGUAAH5yvc2jn8013.jpg

实际文件存储路径下有创建好的多级目录。data下有256个1级目录，每级目录下又有256个2级子目录，总共65536个文件，新写的文件会以hash的方式被路由到其中某个子目录下，然后将文件数据直接作为一个本地文件存储到该目录中。

如果要访问刚上传的图片，我们得需要结合nginx来实现

五、安装Nginx并实现配置

1.安装Nginx依赖环境

[root@localhost fdfs]# yum -y install pcre pcre-devel  
[root@localhost fdfs]# yum -y install zlib zlib-devel  
[root@localhost fdfs]# yum -y install openssl openssl-devel

2.安装nginx并添加fastdfs-nginx-module

解压nginx和fastdfs-nginx-module

[root@localhost fdfs]# tar -zxvf nginx-1.10.3.tar.gz
[root@localhost fdfs]# unzip fastdfs-nginx-module-master.zip

解压后进入nginx目录编译安装nginx,并添加fastdfs-nginx-module

[root@localhost nginx-1.10.3]# ./configure --prefix=/usr/local/nginx --add-module=/home/zhangyongliang/apps/fastdfs-nginx-module-master/src #解压后fastdfs-nginx-module所在的位置

之后进行编译和安装

[root@localhost nginx-1.10.3]# make && make isntall

安装成功后，nginx会安装在/usr/local/nginx,安装后查看

[root@localhost src]# ll /usr/local/nginx/
总用量 8
drwx------. 2 nobody root    6 1月  17 13:23 client_body_temp
drwxr-xr-x. 2 root   root 4096 1月  17 13:17 conf
drwx------. 2 nobody root    6 1月  17 13:23 fastcgi_temp
drwxr-xr-x. 2 root   root   40 1月  17 13:17 html
drwxr-xr-x. 2 root   root   58 1月  17 13:49 logs
-rw-r--r--. 1 root   root 1156 1月  17 13:29 nginx.conf
drwx------. 2 nobody root    6 1月  17 13:23 proxy_temp
drwxr-xr-x. 2 root   root   19 1月  17 13:17 sbin
drwx------. 2 nobody root    6 1月  17 13:23 scgi_temp
drwx------. 2 nobody root    6 1月  17 13:23 uwsgi_temp

安装成功后，nginx尚未运行时，nginx文件夹没有临时文件夹，例如fastcgi_temp这些文件。

3.配置Storage Nginx

修改Nginx 目录下 conf 的配置文件nginx.conf,新增location信息，具体如下：

server {
        listen       9991;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
        }

        location ~/group1/M00 {
            root /opt/fastdfs_storage/data;
            ngx_fastdfs_module;
        }

        location = /50x.html {
            root   html;
        }
}

然后进入FastDFS安装时的解压过的目录，将http.conf和mime.types拷贝到/etc/fdfs目录下：

[root@localhost src]# cp http.conf  mime.types  /etc/fdfs/
[root@localhost src]# cp mime.types  /etc/fdfs/

另外还需要把fastdfs-nginx-module安装目录中src目录下的mod_fastdfs.conf也拷贝到/etc/fdfs目录下：

[root@localhost src]# cp mod_fastdfs.conf  /etc/fdfs/

对刚刚拷贝的mod_fastdfs.conf文件进行修改：

base_path=/opt/fastdfs_storage  #保存日志目录
tracker_server=192.168.43.60:22122 #tracker服务器的IP地址以及端口号
storage_server_port=23000 #storage服务器的端口号
url_have_group_name = true #文件 url 中是否有 group 名
store_path0=/opt/fastdfs_storage_data   #存储路径
group_count = 1 #设置组的个数

在文件的最后，设置group

[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/opt/fastdfs_storage_data

创建M00至storage存储目录的符号连接：

ln  -s  /opt/fastdfs_storage_data/data/ /opt/fastdfs_storage_data/data/M00

启动Nginx：

[root@localhost src]# /usr/local/nginx/sbin/nginx

访问Nginx是否启动

[root@localhost src]# curl localhost:9991







Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
nginx.org.

Commercial support is available at
nginx.com.

Thank you for using nginx.

4.配置Tracker Nginx

在nginx.conf 文件添加一个虚拟机

upstream fdfs_group1 {
        server 127.0.0.1:9991;
    }
   server {
        listen       80;
        server_name  localhost;
       location /group1/M00 {
            proxy_pass http://fdfs_group1;
        }
      error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

完整nginx.conf配置文件如下：

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    server {
        listen       9991;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }
        location ~/group1/M00 {
            root /opt/fastdfs_storage/data;
            ngx_fastdfs_module;
        }
        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }

    upstream fdfs_group1 {
       server 127.0.0.1:9991;
    }
     server {
        listen       80;
        server_name  localhost;
       location /group1/M00 {
            proxy_pass http://fdfs_group1;
        }
      error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}


    # HTTPS server
    #
    #server {
    #    listen       443 ssl;
    #    server_name  localhost;

    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;

    #    ssl_session_cache    shared:SSL:1m;
    #    ssl_session_timeout  5m;

    #    ssl_ciphers  HIGH:!aNULL:!MD5;
    #    ssl_prefer_server_ciphers  on;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

}

将Nginx重新启动

[root@localhost src]# /usr/local/nginx/sbin/nginx -s reload

访问Nginx是否已经启动

[root@localhost src]# curl localhost:9991







Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
nginx.org.

Commercial support is available at
nginx.com.

Thank you for using nginx.


[root@localhost src]# curl localhost







Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
nginx.org.

Commercial support is available at
nginx.com.

Thank you for using nginx.

5.HTTP访问图片资源

访问图片资源路径http://192.168.43.60/group1/M00/00/00/wKgrPFpeyM2ATkGUAAH5yvc2jn8013.jpg

FastDFS资源访问截图

说明： 如图片资源无法访问，请关闭系统防火墙
可以看到已经图片，说明你已成功实现了FastDFS单机的文件系统搭建。

补充说明：

如果Tracker 服务、Storage服务、Nginx服务开机后没有自启动，请执行一下操作并进行重启

[root@localhost ~]# chkconfig  --add fdfs_trackerd
[root@localhost ~]# chkconfig  --add fdfs_storaged

编辑目录下的/etc/rc.d/rc.local,内容如下：

#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

touch /var/lock/subsys/local
/usr/local/nginx/sbin/nginx

主要增加了Nginx的启动，之后进行文件授权，生效文件，重新启动系统

[root@localhost ~]# chmod  +x /etc/rc.d/rc.local
[root@localhost ~]# source /etc/rc.d/rc.local 
[root@localhost ~]# reboot