简介
SeaweedFS是一个简单并且高度可扩展的分布式文件系统,可以存储数十亿的文件并且快速获得文件,特别适合于有效处理小文件概念:
备份策略
weed-fs提供了若干种replication策略(rack – 机架,一个逻辑上的概念):
000 no replication, just one copy
001 replicate once on the same rack
010 replicate once on a different rack in the same data center
100 replicate once on a different data center
200 replicate twice on two other different data center
110 replicate once on a different rack, and once on a different data center
集群模式:注意选择的备份策略,不同的策略需要不同的数据中心机器。否则会报“Cannot grow volume group! No enough data node found” 错误
master: earth139, earth140, earth141(三台机器)
./weed master -ip 10.141.160.93 -port 9333 -mdir ./weedfs_master -peers 10.141.160.93:9333,10.141.160.94:9333,10.141.160.97:9333 -defaultReplication 100 >master.log 2>&1 &
./weed master -ip 10.141.160.94 -port 9333 -mdir ./weedfs_master -peers 10.141.160.93:9333,10.141.160.94:9333,10.141.160.97:9333 -defaultReplication 100 >master.log 2>&1 &
./weed master -ip 10.141.160.97 -port 9333 -mdir ./weedfs_master -peers 10.141.160.93:9333,10.141.160.94:9333,10.141.160.97:9333 -defaultReplication 100 >master.log 2>&1 &
volume:earth139, earth140, earth141
./weed volume -ip 10.141.160.93 -port 9222 -dir ./weedfs_volume -mserver 10.141.160.93:9333,10.141.160.94:9333,10.141.160.97:9333 -dataCenter dc1 -rack rack1 >volume.log 2>&1 &
./weed volume -ip 10.141.160.94 -port 9222 -dir ./weedfs_volume -mserver 10.141.160.93:9333,10.141.160.94:9333,10.141.160.97:9333 -dataCenter dc1 -rack rack1 >volume.log 2>&1 &
./weed volume -ip 10.141.160.97 -port 9222 -dir ./weedfs_volume -mserver 10.141.160.93:9333,10.141.160.94:9333,10.141.160.97:9333 -dataCenter dc2 -rack rack1 >volume.log 2>&1 &
filer: 文件服务
./weed scaffold -config filer -output="." // 生成 filer.toml 文件
./weed filer -master 10.141.160.93:9333,10.141.160.94:9333,10.141.160.95:9333 -port 8888
浏览器访问 http://10.141.160.93:8888/
./weed mount -filer="10.141.160.93:8888" -dir="./mountp_seaweed/"
filer server都是工作在standalone模式下的。可使用redis或Cassandra作为后端,同步,需要在
存文件
curl -F [email protected] http://10.141.160.93:9333/submit
{"eTag":"8ffdca8b","fid":"1,02fd956ac2","fileName":"master.log","fileUrl":"10.141.160.94:9222/1,02fd956ac2","size":1165}
也可以请求一个fid,再传
curl -X POST http://10.141.160.93:9333/dir/assign
{"fid":"2,0194c2cb41","url":"10.141.160.94:9222","publicUrl":"10.141.160.94:9222","count":1}
curl -X PUT -F [email protected] http://10.141.160.94:9222/2,0194c2cb41
获取
curl http://10.141.160.94:9222/1,02fd956ac2
或者浏览器打开 http://10.141.160.94:9222/1,02fd956ac2
删除
curl -X DELETE http://10.141.160.94:9222/1,02fd956ac2
垃圾回收
一般删除后,卷的大小并不会马上更新,当然可手动发起更新大小操作
curl "http://10.141.160.93:9333/vol/vacuum"
参考
https://tonybai.com/2015/08/22/intro-of-using-weedfs/
https://blog.csdn.net/DPnice/article/details/84990050
https://www.jianshu.com/p/8adda9b8856e
https://cloud.tencent.com/developer/article/1363021
https://blog.yasking.org/a/weed-fs.html
https://github.com/bingoohuang/blog/issues/57