百度云存储,即BCS(Baidu Cloud Storage),提供object网络存储服务,旨在利用百度在分布式以及网络方面的优势为开发者提供安全、简单、高效的存储服务。
百度云存储提供了一系列简单易用的REST API接口、SDK、工具和方案,使得开发者可以在任何时间、任何地点存储任何类型的数据。通过使用百度云存储服务,开发者可以轻松地开发出扩展性强、稳定性好、安全快速的分布式网络服务;通过使用云存储服务提供的API、SDK及管理平台,开发者也可以迅速开发出适合各种业务的网络程序。
百度云存储可以支持文本、多媒体、二进制等任何类型的数据,支持签名认证及ACL权限设置进行资源访问控制,开发者可以通过管理控制台直接进行页面上传、下载或通过REST API、Shell Tool、SDK、curl等方式实现上传、下载。
百度云存储服务主要提供以下功能供开发者使用:
功能 | 描述 | |
---|---|---|
1 | 支持任何类型数据的上传和下载 | 文本、多媒体、日志、二进制等类型数据 |
2 | 强大的元信息机制 | 开发者可以使用通用的和自定义的元信息机制来定义资源属性。 |
3 | 超大容量 | 云存储支持0-2T的单文件数据容量,同时对object的个数没有限制;云存储提供superfile接口,可以实现2T超大文件的上传和下载。 |
4 | 分片上传、断点下载功能。 | 该功能在网络不稳定的环境下具有非常好的表现。 |
5 | Restful风格的HTTP接口和多语言SDK及工具,可极大提高开发者的开发效率。 | 了解RESTful的相关信息,可参考http://en.wikipedia.org/wiki/Representational_State_Transfer。 |
6 | 签名认证方案 | 基于公钥和密钥的认证方案可适应灵活的业务需求。 |
7 | 强大的ACL权限控制 | 开发者可通过ACL设置资源的权限(公开或私有),也可授权特定用户具有特定权限。 |
8 | 功能完善的管理平台 | 开发者可通过管理中心对所有资源进行统一管理。 |
9 | 静态网页托管 | 开发者可通过将静态页面上传至云存储空间中,并将其访问权限设置为公开读,即可实现静态网站的效果。 |
详见:http://developer.baidu.com/wiki/index.php?title=docs/cplat/bcs
本文主要面向百度云存储服务的开发者,开发者可通过此文学习如何使用百度云存储进行开发。
使用百度云存储服务进行开发的具体步骤如下:
使用云存储服务,需要首先创建应用并获取应用密钥对。操作步骤如下:
1. 在管理中心创建一个应用
2. 在应用基本信息页,获取相应的“API key 及 Secret Key”
创建Bucket,可通过以下三种方式来实现:
使用云存储服务上传文件有以下四种途径:
云存储bucket可以看成是命名空间,其名称全局唯一,如果某个名称已创建,则其他开发者将无法再创建相同名称的bucket;开发者上传的任何数据必须是属于某个bucket。上传任何数据之前需先确认是否已经创建bucket。 有关bucket的命名限制,请参考#约束与限制。 创建bucket有以下两种方式:
Object是BCS中的基本存储实体,由Key(键)、Data(数据)及MetaData (元数据)组成。
指object的唯一标识,其命名必须以“/”开头,有关object的命名限制,请参考#约束与限制。 如: mystorage/test/我的文件.txt
普通文件的文件大小最大为2GB, 对于更大的超大文件(最大支持2TB)可通过put superfile API来实现。
**
*
* @param baiduBCS
* @param bucket 仓库名
* @param object 上传到仓库中的对象
* @param destFile 本地需要上传的文件对象
* @param contentType 文件类型
*/
public static void putObjectByFile(BaiduBCS baiduBCS,String bucket,String object,File destFile,String contentType) {
PutObjectRequest request = new PutObjectRequest(bucket, object, destFile);
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentType(contentType);
request.setMetadata(metadata);
BaiduBCSResponse response = baiduBCS.putObject(request);
ObjectMetadata objectMetadata = response.getResult();
log.info("x-bs-request-id: " + response.getRequestId());
log.info(objectMetadata);
}
/**
* 修改文件的权限
* @param baiduBCS 存储上下文
* @param bucket 存储仓库
* @param object 文件对象
* @param acl 控制权限
*/
public static void putObjectPolicyByX_BS_ACL(BaiduBCS baiduBCS,String bucket ,String object,X_BS_ACL acl) {
baiduBCS.putObjectPolicy(bucket, object, acl);
}
4.0.0
baiduBCS
baiduBCS
1.0-SNAPSHOT
log4j
log4j
1.2.17
net.sf.flexjson
flexjson
2.1
org.apache.httpcomponents
httpclient
4.3.1
commons-codec
commons-codec
1.8
import com.baidu.inf.iis.bcs.BaiduBCS;
import com.baidu.inf.iis.bcs.model.X_BS_ACL;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Properties;
/**
* Created with IntelliJ IDEA.
* User: liu.xing
* Date: 14-1-20
* Time: 下午7:23
* coding for fun and coding my life!
*/
public class MultipUpload {
static String filePrefix = "E:/站点/innos";
static int len = filePrefix.length();
private static ArrayList filelist = new ArrayList();
static String bucket = "innosrom"; //文件仓库名
static String bcsDir = "/webDesgin/01/"; //文件存放目录
static String object = ""; //文件对象本身
static BaiduBCS bcs;
static Properties pro;
public static void main(String[] args) {
MultipUpload mu = new MultipUpload();
long a = System.currentTimeMillis();
pro = mu.loadFileContentType(); //载入文件格式对应的文档类型
bcs = Sample.init(); //初始化百度云存储环境
mu.refreshFileList(filePrefix); //遍历文件
System.out.println(System.currentTimeMillis() - a);
}
/**
* 遍历文件
* @param strPath 文件夹路径
*/
public void refreshFileList(String strPath) {
File dir = new File(strPath);
File[] files = dir.listFiles();
if (files == null)
return;
for (int i = 0; i < files.length; i++) {
String absPath = files[i].getAbsolutePath();
if (files[i].isDirectory()) {
refreshFileList(absPath);
} else {
String strFileName = absPath.replace("\\", "/");
object = bcsDir + strFileName.substring(len + 1);
String contentType = getContentType(object,pro);
System.out.println(object+"\t"+contentType);
Sample.putObjectByFile(bcs, bucket, object, new File(absPath),contentType);
Sample.putObjectPolicyByX_BS_ACL(bcs, bucket, object, X_BS_ACL.PublicReadWrite);
//deleteAll(bcs,object);
}
}
}
/**
* 根据文件名获取ContentType
* @param object 文件名
* @param pro contentType-file 对应的 Properties对象
* @return ContentType
*/
private String getContentType(String object,Properties pro) {
if(object.contains(".")){
int index = object.lastIndexOf(".");
String suffix = object.substring(index+1); //获取文件后缀
Object contentType = pro.get(suffix);
if(contentType != null)
return contentType.toString();
}
return "text/html";
}
public void deleteAll(BaiduBCS bcs,String object){
Sample.deleteObject(bcs,object);
}
/**
* 载入ContentType 文件
* @return
*/
public Properties loadFileContentType(){
InputStream is = getClass().getClassLoader().getSystemResourceAsStream("contentType.proerties");
Properties pro = new Properties();
try {
pro.load(is);
} catch (IOException e) {
e.printStackTrace();
}
return pro;
}
}
/***************************************************************************
*
* Copyright (c) 2012 Baidu.com, Inc. All Rights Reserved
*
**************************************************************************/
import java.io.*;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.List;
import com.baidu.inf.iis.bcs.model.*;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import com.baidu.inf.iis.bcs.BaiduBCS;
import com.baidu.inf.iis.bcs.auth.BCSCredentials;
import com.baidu.inf.iis.bcs.auth.BCSSignCondition;
import com.baidu.inf.iis.bcs.http.HttpMethodName;
import com.baidu.inf.iis.bcs.policy.Policy;
import com.baidu.inf.iis.bcs.policy.PolicyAction;
import com.baidu.inf.iis.bcs.policy.PolicyEffect;
import com.baidu.inf.iis.bcs.policy.Statement;
import com.baidu.inf.iis.bcs.request.CreateBucketRequest;
import com.baidu.inf.iis.bcs.request.GenerateUrlRequest;
import com.baidu.inf.iis.bcs.request.GetObjectRequest;
import com.baidu.inf.iis.bcs.request.ListBucketRequest;
import com.baidu.inf.iis.bcs.request.ListObjectRequest;
import com.baidu.inf.iis.bcs.request.PutObjectRequest;
import com.baidu.inf.iis.bcs.request.PutSuperfileRequest;
import com.baidu.inf.iis.bcs.response.BaiduBCSResponse;
@SuppressWarnings("unused")
public class Sample {
private static final Log log = LogFactory.getLog(Sample.class);
// ----------------------------------------
static String host = "bcs.duapp.com";
static String accessKey = "您自己的私钥";
static String secretKey = "您自己的密钥";
static String bucket = "innosrom";
// ----------------------------------------
static String object = "/index.jsp";
static File destFile = new File("C:\\BCS\\up.txt");
static File saveFile = new File("C:\\BCS\\download.txt");
public static BaiduBCS init(){
BCSCredentials credentials = new BCSCredentials(accessKey, secretKey);
BaiduBCS baiduBCS = new BaiduBCS(credentials, host);
// baiduBCS.setDefaultEncoding("GBK");
baiduBCS.setDefaultEncoding("UTF-8"); // Default UTF-8
return baiduBCS;
}
/**
* @param args
* @throws java.net.URISyntaxException
* @throws java.io.IOException
*/
public static void main(String[] args) throws URISyntaxException, IOException {
// PropertyConfigurator.configure("log4j.properties");
BCSCredentials credentials = new BCSCredentials(accessKey, secretKey);
BaiduBCS baiduBCS = new BaiduBCS(credentials, host);
// baiduBCS.setDefaultEncoding("GBK");
baiduBCS.setDefaultEncoding("UTF-8"); // Default UTF-8
try {
// -------------bucket-------------
// listBucket(baiduBCS);
// createBucket(baiduBCS);
// deleteBucket(baiduBCS);
// getBucketPolicy(baiduBCS);
// putBucketPolicyByPolicy(baiduBCS);
// putBucketPolicyByX_BS_ACL(baiduBCS, X_BS_ACL.PublicControl);
// listObject(baiduBCS);
// ------------object-------------
//putObjectByFile(baiduBCS);
//putObjectByInputStream(baiduBCS);
//getObjectWithDestFile(baiduBCS);
// putSuperfile(baiduBCS);
//deleteObject(baiduBCS);
//getObjectMetadata(baiduBCS);
// setObjectMetadata(baiduBCS);
// copyObject(baiduBCS, bucket, object + "_copy" +
// (System.currentTimeMillis()));
//getObjectPolicy(baiduBCS);
//putObjectPolicyByPolicy(baiduBCS);
putObjectPolicyByX_BS_ACL(baiduBCS, X_BS_ACL.PublicReadWrite);
// ------------common------------------
//generateUrl( baiduBCS);
} catch (BCSServiceException e) {
log.warn("Bcs return:" + e.getBcsErrorCode() + ", " + e.getBcsErrorMessage() + ", RequestId=" + e.getRequestId());
} catch (BCSClientException e) {
e.printStackTrace();
}
}
public static void generateUrl(BaiduBCS baiduBCS) {
GenerateUrlRequest generateUrlRequest = new GenerateUrlRequest(HttpMethodName.GET, bucket, object);
generateUrlRequest.setBcsSignCondition(new BCSSignCondition());
generateUrlRequest.getBcsSignCondition().setIp("*");
//generateUrlRequest.getBcsSignCondition().setTime(123455L);
//generateUrlRequest.getBcsSignCondition().setSize(123455L);
System.out.println(baiduBCS.generateUrl(generateUrlRequest));
}
public static void copyObject(BaiduBCS baiduBCS, String destBucket, String destObject) {
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setContentType("image/jpeg");
baiduBCS.copyObject(new Resource(bucket, object), new Resource(destBucket, destObject), objectMetadata);
baiduBCS.copyObject(new Resource(bucket, object), new Resource(destBucket, destObject), null);
baiduBCS.copyObject(new Resource(bucket, object), new Resource(destBucket, destObject));
}
private static void createBucket(BaiduBCS baiduBCS) {
// baiduBCS.createBucket(bucket);
baiduBCS.createBucket(new CreateBucketRequest(bucket, X_BS_ACL.PublicRead));
}
private static void deleteBucket(BaiduBCS baiduBCS) {
baiduBCS.deleteBucket(bucket);
}
public static void deleteObject(BaiduBCS baiduBCS,String object) {
Empty result = baiduBCS.deleteObject(bucket, object).getResult();
log.info(result);
}
private static void getBucketPolicy(BaiduBCS baiduBCS) {
BaiduBCSResponse response = baiduBCS.getBucketPolicy(bucket);
log.info("After analyze: " + response.getResult().toJson());
log.info("Origianal str: " + response.getResult().getOriginalJsonStr());
}
public static void getObjectMetadata(BaiduBCS baiduBCS) {
ObjectMetadata objectMetadata = baiduBCS.getObjectMetadata(bucket, object).getResult();
log.info(objectMetadata);
}
private static void getObjectPolicy(BaiduBCS baiduBCS) {
BaiduBCSResponse response = baiduBCS.getObjectPolicy(bucket, object);
log.info("After analyze: " + response.getResult().toJson());
log.info("Origianal str: " + response.getResult().getOriginalJsonStr());
}
private static void getObjectWithDestFile(BaiduBCS baiduBCS) {
GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, object);
baiduBCS.getObject(getObjectRequest, saveFile);
}
private static void listBucket(BaiduBCS baiduBCS) {
ListBucketRequest listBucketRequest = new ListBucketRequest();
BaiduBCSResponse> response = baiduBCS.listBucket(listBucketRequest);
for (BucketSummary bucket : response.getResult()) {
log.info(bucket);
}
}
private static void listObject(BaiduBCS baiduBCS) {
ListObjectRequest listObjectRequest = new ListObjectRequest(bucket);
listObjectRequest.setStart(0);
listObjectRequest.setLimit(20);
// ------------------by dir
{
// prefix must start with '/' and end with '/'
// listObjectRequest.setPrefix("/1/");
// listObjectRequest.setListModel(2);
}
// ------------------only object
{
// prefix must start with '/'
// listObjectRequest.setPrefix("/1/");
}
BaiduBCSResponse response = baiduBCS.listObject(listObjectRequest);
log.info("we get [" + response.getResult().getObjectSummaries().size() + "] object record.");
for (ObjectSummary os : response.getResult().getObjectSummaries()) {
log.info(os.toString());
}
}
private static void putBucketPolicyByPolicy(BaiduBCS baiduBCS) {
Policy policy = new Policy();
Statement st1 = new Statement();
st1.addAction(PolicyAction.all).addAction(PolicyAction.get_object);
st1.addUser("zhengkan").addUser("zhangyong01");
st1.addResource(bucket + "/111").addResource(bucket + "/111");
st1.setEffect(PolicyEffect.allow);
policy.addStatements(st1);
baiduBCS.putBucketPolicy(bucket, policy);
}
private static void putBucketPolicyByX_BS_ACL(BaiduBCS baiduBCS, X_BS_ACL acl) {
baiduBCS.putBucketPolicy(bucket, acl);
}
public static void putObjectByFile(BaiduBCS baiduBCS) {
PutObjectRequest request = new PutObjectRequest(bucket, object, destFile);
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentType("text/html");
request.setMetadata(metadata);
BaiduBCSResponse response = baiduBCS.putObject(request);
ObjectMetadata objectMetadata = response.getResult();
log.info("x-bs-request-id: " + response.getRequestId());
log.info(objectMetadata);
}
/**
*
* @param baiduBCS
* @param bucket 仓库名
* @param object 上传到仓库中的对象
* @param destFile 本地需要上传的文件对象
* @param contentType 文件类型
*/
public static void putObjectByFile(BaiduBCS baiduBCS,String bucket,String object,File destFile,String contentType) {
PutObjectRequest request = new PutObjectRequest(bucket, object, destFile);
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentType(contentType);
request.setMetadata(metadata);
BaiduBCSResponse response = baiduBCS.putObject(request);
ObjectMetadata objectMetadata = response.getResult();
log.info("x-bs-request-id: " + response.getRequestId());
log.info(objectMetadata);
}
public static void putObjectByInputStream(BaiduBCS baiduBCS) throws FileNotFoundException {
File file = createSampleFile();
InputStream fileContent = new FileInputStream(file);
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setContentType("text/html");
objectMetadata.setContentLength(file.length());
PutObjectRequest request = new PutObjectRequest(bucket, object, fileContent, objectMetadata);
ObjectMetadata result = baiduBCS.putObject(request).getResult();
log.info(result);
}
private static void putObjectPolicyByPolicy(BaiduBCS baiduBCS) {
Policy policy = new Policy();
Statement st1 = new Statement();
st1.addAction(PolicyAction.all).addAction(PolicyAction.get_object);
st1.addUser("zhengkan").addUser("zhangyong01");
st1.addResource(bucket + object).addResource(bucket + object);
st1.setEffect(PolicyEffect.allow);
policy.addStatements(st1);
baiduBCS.putObjectPolicy(bucket, object, policy);
}
private static void putObjectPolicyByX_BS_ACL(BaiduBCS baiduBCS, X_BS_ACL acl) {
baiduBCS.putObjectPolicy(bucket, object, acl);
}
/**
* 修改文件的权限
* @param baiduBCS 存储上下文
* @param bucket 存储仓库
* @param object 文件对象
* @param acl 控制权限
*/
public static void putObjectPolicyByX_BS_ACL(BaiduBCS baiduBCS,String bucket ,String object,X_BS_ACL acl) {
baiduBCS.putObjectPolicy(bucket, object, acl);
}
public static void putSuperfile(BaiduBCS baiduBCS) {
List subObjectList = new ArrayList();
// 0
BaiduBCSResponse response1 = baiduBCS.putObject(bucket, object + "_part0", createSampleFile());
subObjectList.add(new SuperfileSubObject(bucket, object + "_part0", response1.getResult().getETag()));
// 1
BaiduBCSResponse response2 = baiduBCS.putObject(bucket, object + "_part1", createSampleFile());
subObjectList.add(new SuperfileSubObject(bucket, object + "_part1", response2.getResult().getETag()));
// put superfile
PutSuperfileRequest request = new PutSuperfileRequest(bucket, object + "_superfile", subObjectList);
BaiduBCSResponse response = baiduBCS.putSuperfile(request);
ObjectMetadata objectMetadata = response.getResult();
log.info("x-bs-request-id: " + response.getRequestId());
log.info(objectMetadata);
}
public static void setObjectMetadata(BaiduBCS baiduBCS) {
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setContentType("text/html12");
baiduBCS.setObjectMetadata(bucket, object, objectMetadata);
}
private static File createSampleFile() {
try {
File file = File.createTempFile("java-sdk-", ".txt");
file.deleteOnExit();
Writer writer = new OutputStreamWriter(new FileOutputStream(file));
writer.write("01234567890123456789\n");
writer.write("01234567890123456789\n");
writer.write("01234567890123456789\n");
writer.write("01234567890123456789\n");
writer.write("01234567890123456789\n");
writer.close();
return file;
} catch (IOException e) {
log.error("tmp file create failed.");
return null;
}
}
}