分布式文件系统FastDFS配置与使用
FastDFS 是一个用C语言编写的分布式文件系统，刚开始接触FastDFS的时候，是因为文件同步。后来发现自己完全理解错误，FastDFS并不是Rsync那样文件同步，但是既然开始了就继续深入了解她。FastDFS支持的文件同步是把文件上传到调度器（tracker），然后按照管理员设定把文件下发到其中一台文件存储器（storage），成功后把文件同步到其他各台机器。FastDFS也可以作为文件存储、文件上传下载等，可以解决文件服务集群的分布式部署与负载均衡。一个典型的例子是充当图片服务器，为论坛、电商等提供高速可靠的图片服务。FastDFS 客户端开发也支持多语言（C、python、java、php等）API，为项目开发提供便利。
1、系统环境
三台Centos_X64_6.5,一个tracker，两个storage，其中一台机器也模仿client的角色，FastDFS源码安装5.05版本。注意：三台机器都要安装FastDFS，而根据对应所在系统的角色配置相关配置文件，然后启动fdfs_trackerd、fdfs_storaged服务。
2、FastDFS安装过程
以源码方式安装，需要提前安装C语言开发环境，还要安装cmake环境等，使用yum命令安装以上开发工具也比较方便，如：yum install cmake gcc git。
2.1安装依赖库
安装libevent命令行输入下命令完成安装：[root@localhost Desktop]# mkdir FastNFS [root@localhost Desktop]# cd FastNFS/ [root@localhost FastNFS]# git clone https://github.com/libevent/libevent.git [root@localhost libevent]# rpm -qa libevent [root@localhost libevent]# rpm -qa|grep libevent rpm -e libevent* [root@localhost FastNFS]# cd libevent [root@localhost libevent]# cd cmake [root@localhost cmake]# yum install cmake [root@localhost cmake]# cmake .. [root@localhost cmake]# make [root@localhost cmake]# make install
安装libfastcommon，命令行输入下命令完成安装：
[root@localhost FastNFS]# git clone https://github.com/happyfish100/libfastcommon.git [root@localhost FastNFS]# cd libfastcommon [root@localhost libfastcommon]# ./make.sh [root@localhost libfastcommon]# ./make.sh install
打印如下面内容即安装成功：
mkdir -p /usr/lib64 mkdir -p /usr/lib install -m 755 libfastcommon.so /usr/lib64 install -m 755 libfastcommon.so /usr/lib mkdir -p /usr/include/fastcommon install -m 644 common_define.h hash.h chain.h logger.h base64.h shared_func.h pthread_func.h ini_file_reader.h _os_define.h sockopt.h sched_thread.h http_func.h md5.h local_ip_func.h avl_tree.h ioevent.h ioevent_loop.h fast_task_queue.h fast_timer.h process_ctrl.h fast_mblock.h connection_pool.h fast_mpool.h fast_allocator.h fast_buffer.h skiplist.h multi_skiplist.h flat_skiplist.h skiplist_common.h system_info.h fast_blocked_queue.h php7_ext_wrapper.h /usr/include/fastcommon
2.2安装FastDFS
下载源码包、解压、进入目录完成安装：[root@localhost FastNFS]# wget https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gz [root@localhost FastNFS]# tar -zxvf V5.05.tar.gz [root@localhost FastNFS]# cd fastdfs-5.05 [root@localhost fastdfs-5.05]# ./make.sh [root@localhost fastdfs-5.05]# ./make.sh install
打印如下面内容即安装成功：
mkdir -p /usr/bin mkdir -p /etc/fdfs cp -f fdfs_trackerd /usr/bin if [ ! -f /etc/fdfs/tracker.conf.sample ]; then cp -f ../conf/tracker.conf /etc/fdfs/tracker.conf.sample; fi mkdir -p /usr/bin mkdir -p /etc/fdfs cp -f fdfs_storaged /usr/bin if [ ! -f /etc/fdfs/storage.conf.sample ]; then cp -f ../conf/storage.conf /etc/fdfs/storage.conf.sample; fi mkdir -p /usr/bin mkdir -p /etc/fdfs mkdir -p /usr/lib64 cp -f fdfs_monitor fdfs_test fdfs_test1 fdfs_crc32 fdfs_upload_file fdfs_download_file fdfs_delete_file fdfs_file_info fdfs_appender_test fdfs_appender_test1 fdfs_append_file fdfs_upload_appender /usr/bin if [ 0 -eq 1 ]; then cp -f libfdfsclient.a /usr/lib64; fi if [ 1 -eq 1 ]; then cp -f libfdfsclient.so /usr/lib64; fi mkdir -p /usr/include/fastdfs cp -f ../common/fdfs_define.h ../common/fdfs_global.h ../common/mime_file_parser.h ../common/fdfs_http_shared.h ../tracker/tracker_types.h ../tracker/tracker_proto.h ../tracker/fdfs_shared_func.h ../storage/trunk_mgr/trunk_shared.h tracker_client.h storage_client.h storage_client1.h client_func.h client_global.h fdfs_client.h /usr/include/fastdfs if [ ! -f /etc/fdfs/client.conf.sample ]; then cp -f ../conf/client.conf /etc/fdfs/client.conf.sample; fi
查看FastDFS安装后的配置文件：
[root@localhost fastdfs-5.05]# ll /etc/fdfs/
打印输出下面信息：
total 20 -rw-r--r--. 1 root root 1461 Mar 21 03:01 client.conf.sample -rw-r--r--. 1 root root 7829 Mar 21 03:01 storage.conf.sample -rw-r--r--. 1 root root 7102 Mar 21 03:01 tracker.conf.sample
3、FastDFS简单配置
3.1配置tracker
创建配置文件：[root@localhost fastdfs-5.05]# cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
创建base目录
[root@localhost github.com]# mkdir -p /home/yuqing/fastdfs
打开配置文件，主要bind_addr、port、base_path配置几个选项
[root@localhost fastdfs-5.05]# vi /etc/fdfs/tracker.conf
根据说明配置成下面内容：
disabled：是否关闭服务，false为不关闭服务
bind_addr：绑定的IP地址
port：绑定的端口地址
base_path：指定文件存储目录
附上配置文件内容，方便查看解析说明：
# is this config file disabled # false for enabled # true for disabled disabled=false # bind an address of this host # empty for bind all addresses of this host bind_addr= # the tracker server port port=22122 # connect timeout in seconds # default value is 30s connect_timeout=30 # network timeout in seconds # default value is 30s network_timeout=60 # the base path to store data and log files base_path=/home/yuqing/fastdfs # max concurrent connections this server supported max_connections=256 # accept thread count # default value is 1 # since V4.07 accept_threads=1 # work thread count, should <= max_connections # default value is 4 # since V2.00 work_threads=4 # the method of selecting group to upload files # 0: round robin # 1: specify group # 2: load balance, select the max free space group to upload file store_lookup=2 # which group to upload file # when store_lookup set to 1, must set store_group to the group name store_group=group2 # which storage server to upload file # 0: round robin (default) # 1: the first server order by ip address # 2: the first server order by priority (the minimal) store_server=0 # which path(means disk or mount point) of the storage server to upload file # 0: round robin # 2: load balance, select the max free space path to upload file store_path=0 # which storage server to download file # 0: round robin (default) # 1: the source storage server which the current file uploaded to download_server=0 # reserved storage space for system or other applications. # if the free(available) space of any stoarge server in # a group <= reserved_storage_space, # no file can be uploaded to this group. # bytes unit can be one of follows: ### G or g for gigabyte(GB) ### M or m for megabyte(MB) ### K or k for kilobyte(KB) ### no unit for byte(B) ### XX.XX% as ratio such as reserved_storage_space = 10% reserved_storage_space = 10% #standard log level as syslog, case insensitive, value list: ### emerg for emergency ### alert ### crit for critical ### error ### warn for warning ### notice ### info ### debug log_level=info #unix group name to run this program, #not set (empty) means run by the group of current user run_by_group= #unix username to run this program, #not set (empty) means run by current user run_by_user= # allow_hosts can ocur more than once, host can be hostname or ip address, # "*" means match all ip addresses, can use range like this: 10.0.1.[1-15,20] or # host[01-08,20-25].domain.com, for example: # allow_hosts=10.0.1.[1-15,20] # allow_hosts=host[01-08,20-25].domain.com allow_hosts=* # sync log buff to disk every interval seconds # default value is 10 seconds sync_log_buff_interval = 10 # check storage server alive interval seconds check_active_interval = 120 # thread stack size, should >= 64KB # default value is 64KB thread_stack_size = 64KB # auto adjust when the ip address of the storage server changed # default value is true storage_ip_changed_auto_adjust = true # storage sync file max delay seconds # default value is 86400 seconds (one day) # since V2.00 storage_sync_file_max_delay = 86400 # the max time of storage sync a file # default value is 300 seconds # since V2.00 storage_sync_file_max_time = 300 # if use a trunk file to store several small files # default value is false # since V3.00 use_trunk_file = false # the min slot size, should <= 4KB # default value is 256 bytes # since V3.00 slot_min_size = 256 # the max slot size, should > slot_min_size # store the upload file to trunk file when it's size <= this value # default value is 16MB # since V3.00 slot_max_size = 16MB # the trunk file size, should >= 4MB # default value is 64MB # since V3.00 trunk_file_size = 64MB # if create trunk file advancely # default value is false # since V3.06 trunk_create_file_advance = false # the time base to create trunk file # the time format: HH:MM # default value is 02:00 # since V3.06 trunk_create_file_time_base = 02:00 # the interval of create trunk file, unit: second # default value is 38400 (one day) # since V3.06 trunk_create_file_interval = 86400 # the threshold to create trunk file # when the free trunk file size less than the threshold, will create # the trunk files # default value is 0 # since V3.06 trunk_create_file_space_threshold = 20G # if check trunk space occupying when loading trunk free spaces # the occupied spaces will be ignored # default value is false # since V3.09 # NOTICE: set this parameter to true will slow the loading of trunk spaces # when startup. you should set this parameter to true when neccessary. trunk_init_check_occupying = false # if ignore storage_trunk.dat, reload from trunk binlog # default value is false # since V3.10 # set to true once for version upgrade when your version less than V3.10 trunk_init_reload_from_binlog = false # the min interval for compressing the trunk binlog file # unit: second # default value is 0, 0 means never compress # FastDFS compress the trunk binlog when trunk init and trunk destroy # recommand to set this parameter to 86400 (one day) # since V5.01 trunk_compress_binlog_min_interval = 0 # if use storage ID instead of IP address # default value is false # since V4.00 use_storage_id = false # specify storage ids filename, can use relative or absolute path # since V4.00 storage_ids_filename = storage_ids.conf # id type of the storage server in the filename, values are: ## ip: the ip address of the storage server ## id: the server id of the storage server # this paramter is valid only when use_storage_id set to true # default value is ip # since V4.03 id_type_in_filename = ip # if store slave file use symbol link # default value is false # since V4.01 store_slave_file_use_link = false # if rotate the error log every day # default value is false # since V4.02 rotate_error_log = false # rotate error log time base, time format: Hour:Minute # Hour from 0 to 23, Minute from 0 to 59 # default value is 00:00 # since V4.02 error_log_rotate_time=00:00 # rotate error log when the log file exceeds this size # 0 means never rotates log file by log file size # default value is 0 # since V4.02 rotate_error_log_size = 0 # keep days of the log files # 0 means do not delete old log files # default value is 0 log_file_keep_days = 0 # if use connection pool # default value is false # since V4.05 use_connection_pool = false # connections whose the idle time exceeds this time will be closed # unit: second # default value is 3600 # since V4.05 connection_pool_max_idle_time = 3600 # HTTP port on this tracker server http.server_port=8080 # check storage HTTP server alive interval seconds # <= 0 for never check # default value is 30 http.check_alive_interval=30 # check storage HTTP server alive type, values are: # tcp : connect to the storge server with HTTP port only, # do not request and get response # http: storage check alive url must return http status 200 # default value is tcp http.check_alive_type=tcp # check storage HTTP server alive uri/url # NOTE: storage embed HTTP server support uri: /status.html http.check_alive_uri=/status.html
3.2配置storage
创建配置文件：[root@localhost fastdfs-5.05]# cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf
创建base目录：
[root@localhost fastdfs-5.05]# mkdir -p /home/yuqing/fastdfs
主要配置：group_name、bind_addr、base_path、store_path0、tracker_server
[root@localhost fastdfs-5.05]# vi /etc/fdfs/storage.conf
group_name：所在文件组
bind_addr：服务地址
port：服务端口
base_path：文件数据保存与日志目录
store_path0：文件数据保存目录，覆盖base_path的文件数据保存目录配置
tracker_server：调度器的地址端口，IP:Port的形式
其他选项与调度器类是，请继续查看storage的配置文件。
3.3配置client
创建配置文件：[root@localhost fdfs]# cp /etc/fdfs/client.conf.sample client.conf
主要修改：base_path、tracker_server
[root@localhost fdfs]# vi /etc/fdfs/client.conf
配置文件如下：
# connect timeout in seconds # default value is 30s connect_timeout=30 # network timeout in seconds # default value is 30s network_timeout=60 # the base path to store log files base_path=/home/yuqing/fastdfs # tracker_server can ocur more than once, and tracker_server format is # "host:port", host can be hostname or ip address tracker_server=192.168.213.161:22122 #standard log level as syslog, case insensitive, value list: ### emerg for emergency ### alert ### crit for critical ### error ### warn for warning ### notice ### info ### debug log_level=info # if use connection pool # default value is false # since V4.05 use_connection_pool = false # connections whose the idle time exceeds this time will be closed # unit: second # default value is 3600 # since V4.05 connection_pool_max_idle_time = 3600 # if load FastDFS parameters from tracker server # since V4.05 # default value is false load_fdfs_parameters_from_tracker=false # if use storage ID instead of IP address # same as tracker.conf # valid only when load_fdfs_parameters_from_tracker is false # default value is false # since V4.05 use_storage_id = false # specify storage ids filename, can use relative or absolute path # same as tracker.conf # valid only when load_fdfs_parameters_from_tracker is false # since V4.05 storage_ids_filename = storage_ids.conf #HTTP settings http.tracker_server_port=80 #use "#include" directive to include HTTP other settiongs ##include http.conf
4、启动服务
4.1启动调度器
命令行输入：[root@localhost fastdfs-5.05]# fdfs_trackerd /etc/fdfs/tracker.conf
查看日志
[root@localhost fastdfs-5.05]# cat /home/yuqing/fastdfs/logs/trackerd.log
打印如下：
[2016-03-21 23:12:16] INFO - FastDFS v5.05, base_path=/home/yuqing/fastdfs, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=22122, bind_addr=192.168.213.161, max_connections=256, accept_threads=1, work_threads=4, store_lookup=2, store_group=, store_server=0, store_path=0, reserved_storage_space=10.00%, download_server=0, allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval=120s, thread_stack_size=64 KB, storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_file_max_time=300s, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_compress_binlog_min_interval=0, use_storage_id=0, id_type_in_filename=ip, storage_id_count=0, rotate_error_log=0, error_log_rotate_time=00:00, rotate_error_log_size=0, log_file_keep_days=0, store_slave_file_use_link=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s
查看进程：
[root@localhost fastdfs-5.05]# ps -ef |grep track
打印如下：
root 12839 1 0 23:12 ? 00:00:00 fdfs_trackerd /etc/fdfs/tracker.conf root 12884 5571 0 23:15 pts/2 00:00:00 grep track
查看fdfs服务
[root@localhost fastdfs-5.05]# netstat -ntpl |grep fdfs
打印如下：
tcp 0 0 192.168.213.161:22122 0.0.0.0:* LISTEN 12839/fdfs_trackerd
4.2启动文件存储
命令行输入：[root@localhost fastdfs-5.05]# fdfs_storaged /etc/fdfs/storage.conf
查看日志
[root@localhost fastdfs-5.05]# cat /home/yuqing/fastdfs/logs/storaged.log
打印如下：
[2016-03-21 23:31:50] INFO - FastDFS v5.05, base_path=/home/yuqing/fastdfs, store_path_count=1, subdir_count_per_path=256, group_name=group1, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=23000, bind_addr=192.168.213.136, client_bind=1, max_connections=256, accept_threads=1, work_threads=4, disk_rw_separated=1, disk_reader_threads=1, disk_writer_threads=1, buff_size=256KB, heart_beat_interval=30s, stat_report_interval=60s, tracker_server_count=1, sync_wait_msec=50ms, sync_interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, allow_ip_count=-1, file_distribute_path_mode=0, file_distribute_rotate_count=100, fsync_after_written_bytes=0, sync_log_buff_interval=10s, sync_binlog_buff_interval=10s, sync_stat_file_interval=300s, thread_stack_size=512 KB, upload_priority=10, if_alias_prefix=, check_file_duplicate=0, file_signature_method=hash, FDHT group count=0, FDHT server count=0, FDHT key_namespace=, FDHT keep_alive=0, HTTP server port=8888, domain name=, use_access_log=0, rotate_access_log=0, access_log_rotate_time=00:00, rotate_error_log=0, error_log_rotate_time=00:00, rotate_access_log_size=0, rotate_error_log_size=0, log_file_keep_days=0, file_sync_skip_invalid_record=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s data path: /home/yuqing/fastdfs/data, mkdir sub dir... mkdir data path: 00 ... mkdir data path: 01 ... mkdir data path: 02 ... mkdir data path: 03 ... mkdir data path: 04 ... mkdir data path: 05 ... mkdir data path: 06 ...
查看进程：
[root@localhost fastdfs-5.05]# ps -ef |grep fdfs
打印如下：
root 16280 1 0 23:32 ? 00:00:00 fdfs_storaged /etc/fdfs/storage.conf root 16310 1185 0 23:33 pts/2 00:00:00 grep fdfs
查看fdfs服务：
[root@localhost fastdfs-5.05]# netstat -ntpl |grep fdfs
打印如下：
tcp 0 0 192.168.213.136:23000 0.0.0.0:* LISTEN 16280/fdfs_storaged
5、测试FastDFS功能
在配置了client配置文件的机器上面进行上传文件测试，结果应该返回上传文件的保存路径以及文件名，查看两台文件存储器上相同目录下以相同的文件名保存上传的文件。使用client上传文件命令,将返回文件上传结果，文件的保存目录、文件名等信息：
上传文件：
[root@localhost Desktop]# fdfs_upload_file /etc/fdfs/client.conf index.html
group1/M00/00/00/wKjVh1bw-1KADEceAAAsSKZJhlk53.html
验证测试结果，两台文件服务器拥有相同的文件。
storage查看上传文件：
[root@localhost 00]# ll /home/yuqing/fastdfs/data/00/00/
total 24 -rw-r--r--. 1 root root 11336 Mar 22 00:59 wKjVh1bw-1KADEceAAAsSKZJhlk53.html -rw-r--r--. 1 root root 11336 Mar 22 01:05 wKjViFbw_LyAIv-oAAAsSKZJhlk82.html
6、说明
FastDFS自带访问文件服务http功能，在安装的时候有相关配置选项，如果打开了该http服务功能，可以按照client返回的上传文件保存路径以及文件名，在浏览器可以直接访问。