本文是基于golang做的开发,大家注意,其他版本应该类似。
seaweedfs是golang 开发的分布式文件存储系统,特别是小文件的存取做了很多优化,作者chris也是一个相当勤奋的开源大佬,seaweedfs的github地址是:https://github.com/chrislusf/seaweedfs
在chris的努力下release的1.26版本及其以后的版本都开始支持Amazon的S3文件存储规范,既然支持S3那么我们就想使用S3的aws的sdk进行文件的操作,这样我们只需要有一套文件存储的client就可以适应所有的支持S3协议的文件存储服务,可谓是一劳永逸。
废话不多说我们开始吧,本例采用的seaweedfs的版本是最新release的1.30版本,步骤如下:
1.首先要下载weed源码,并编译启动
#注: 此处仅做测试验证使用只启动一个服务,具体使用时需要部署多个master,根据需要部署多个weed的datanode
#进入源码中的docker目录
cd seaweedfs/docker
#启动weed服务
docker-compose -f seaweedfs-compose.yml -p seaweedfs up
#通过dockerfile查看具体的端口使用情况
# s3 server http port
EXPOSE 8333
#从上面的配置可以看到weed的S3的服务端口是8333
#出现以下内容代表启动成功
Starting seaweedfs_master_1 ... done
Starting seaweedfs_volume_1 ... done
Starting seaweedfs_cronjob_1 ... done
Starting seaweedfs_filer_1 ... done
Starting seaweedfs_s3_1 ... done
Attaching to seaweedfs_master_1, seaweedfs_volume_1, seaweedfs_cronjob_1, seaweedfs_filer_1, seaweedfs_s3_1
master_1 | I0417 03:37:19 1 file_util.go:19] Folder /data Permission: -rwxr-xr-x
master_1 | I0417 03:37:19 1 master_server.go:76] Volume Size Limit is 30000 MB
master_1 | I0417 03:37:19 1 master.go:87] Start Seaweed Master 30GB 1.30 at 0.0.0.0:9333
master_1 | I0417 03:37:19 1 raft_server.go:44] Starting RaftServer with 127.0.0.1:9333
master_1 | I0417 03:37:19 1 raft_server.go:84] current cluster leader:
master_1 | I0417 03:37:19 1 master.go:114] Start Seaweed Master 30GB 1.30 grpc server at 0.0.0.0:19333
master_1 | I0417 03:37:20 1 node.go:224] topo adds child DefaultDataCenter
2.通过aws-sdk-go的sdk访问weed服务
#下载aws-sdk-go
go get github.com/aws/aws-sdk-go
#下载比较慢,多等一会儿,下面介绍使用过程直接上代码
3.使用过程,直接上代码
import (
"bytes"
"encoding/base64"
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"io/ioutil"
)
var (
ak = "1" //文件服务分配的账号
sk = "2" //文件服务分配的秘钥
endPoint = "http://127.0.0.1:8333" //上图weed S3服务的地址
region = "test.com" //适用范围
svc *s3.S3
)
func init() {
cres := credentials.NewStaticCredentials(ak, sk, "")
cfg := aws.NewConfig().WithRegion(region).WithEndpoint(endPoint).WithCredentials(cres).WithS3ForcePathStyle(true)
sess, err := session.NewSession(cfg)
if err != nil {
fmt.Println(err)
}
svc = s3.New(sess)
}
func main() {
//创建桶
bucketName := "weed-test-buck" //桶的名称也是存取这个桶下面数据的唯一标识
createBucket(bucketName)
//将图片数据上传到weed文件服务
objectID := "weed-test-003"
dataImage, err := ioutil.ReadFile("a.jpg")
if err != nil {
fmt.Println(err.Error())
}
contentType := "image/jpeg"
putS3Object(dataImage, bucketName, contentType, objectID)
//获取图片
res := getS3Object(bucketName, objectID)
fmt.Println(base64.StdEncoding.EncodeToString(res))
//按照对象key删除对象信息
deleteS3Object(bucketName, objectID)
}
func createBucket(bucketName string) {
input := &s3.CreateBucketInput{
Bucket: aws.String(bucketName),
}
result, err := svc.CreateBucket(input)
fmt.Println(result)
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
case s3.ErrCodeBucketAlreadyExists:
fmt.Println(s3.ErrCodeBucketAlreadyExists, aerr.Error())
case s3.ErrCodeBucketAlreadyOwnedByYou:
fmt.Println(s3.ErrCodeBucketAlreadyOwnedByYou, aerr.Error())
default:
fmt.Println(aerr.Error())
}
} else {
fmt.Println(err.Error())
}
}
}
func putS3Object(dataImage []byte, bucketName, contentType, objectID string) {
inputObject := &s3.PutObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectID),
ContentType: aws.String(contentType),
Body: bytes.NewReader(dataImage),
}
resp, err := svc.PutObject(inputObject)
if err != nil {
fmt.Println(err.Error())
}
fmt.Println(resp)
}
func getS3Object(bucketName, objectID string) []byte {
inputObject := &s3.GetObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectID),
}
out, err := svc.GetObject(inputObject)
if err != nil {
fmt.Println(err.Error())
return nil
}
res, err := ioutil.ReadAll(out.Body)
if err != nil {
fmt.Println(err.Error())
return nil
}
return res
}
func deleteS3Object(bucketName, objectID string) {
params := &s3.DeleteObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectID),
}
resp, err := svc.DeleteObject(params)
if err != nil {
fmt.Println(err.Error())
}
fmt.Println(resp)
}
查看上传的图片信息可以通过在浏览器中输入一下地址即可获取:
#weed s3的地址+ bucketName + objectID
http://127.0.0.1:8333/weed-test-buck/weed-test-003
以上就是全部内容,seaweedfs目前没有实现policy和acl所以目前初始化时的两个账号信息随便填写但不能为空,不然会报如下错误。
EmptyStaticCreds: static credentials are empty
本人才疏学浅,如有纰漏欢迎指点。
chris做了说明,参看:https://github.com/chrislusf/seaweedfs/wiki/Amazon-S3-API