S3 client 访问ceph rgw
安装：
yum install s3cmd验证安装是否成功：
$s3cmd --version
s3cmd version 1.5.2 #表示安装成功在ceph管理主机上创建S3用户：
sudo radosgw-admin user create --uid="test" --display-name="zhangsan"
查看用户：
sudo radosgw-admin user info --uid="test"
生成json格式的用户信息：
{
"user_id": "test",
"display_name": "zhangjian",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [
{
"id": "test:swift",
"permissions": "full-control"
}
],
"keys": [
{
"user": "test",
"access_key": "K770VAKJYC9PB0O9A113",
"secret_key": "Y1P8tZWsrul1ZOTMPqCiZqNMh13a1IGRxtgYC14f"
}
],
"swift_keys": [
{
"user": "test:swift",
"secret_key": "uoMo0ZYb9xlYanLAeGrzTQlT0ZBn8K6FaODTinKh"
}
],
"caps": [],
"op_mask": "read, write, delete",
"default_placement": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
},
"temp_url_keys": []
}
S3 中access_key和secret_key分别扮演者用户名ID和用户密码的角色。
access_key–>用户名ID
secret_key–>用户密码
配置S3 client：
$ s3cmd --configure Enter new values or accept defaults in brackets with Enter. Refer to user manual for detailed description of all options. Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables. Access Key []: 3WD7VXGJ8ZO0UBTFM5XR Secret Key []: 11111111 Default Region [US]: Encryption password is used to protect your files from reading by unauthorized persons while in transfer to S3 Encryption password: Path to GPG program [/usr/bin/gpg]: When using secure HTTPS protocol all communication with Amazon S3 servers is protected from 3rd party eavesdropping. This method is slower than plain HTTP, and can only be proxied with Python 2.7 or newer Use HTTPS protocol [No]: On some networks all internet access must go through a HTTP proxy. Try setting it here if you can't connect to S3 directly HTTP Proxy server name: New settings: Access Key: 3WD7VXGJ8ZO0UBTFM5XR Secret Key: 11111111 Default Region: US Encryption password: Path to GPG program: /usr/bin/gpg Use HTTPS protocol: False HTTP Proxy server name: HTTP Proxy server port: 0 Test access with supplied credentials? [Y/n] n Save settings? [y/N] y Configuration saved to '/root/.s3cfg'
在这个交互配置过程中，只配置了其中access_key和secret_key，如果要正常使用自己搭建的存储还有3项需要配置：
-
- cloudfont_host
- host_base
- host_bucket
其对应的配置如下：
$cat .s3cfg [default] access_key = K770VAKJYC9PB0O9A113 access_token = add_encoding_exts = add_headers = bucket_location = US ca_certs_file = cache_file = check_ssl_certificate = True check_ssl_hostname = True cloudfront_host = cephcloud.com default_mime_type = binary/octet-stream delay_updates = False delete_after = False delete_after_fetch = False delete_removed = False dry_run = False enable_multipart = True encoding = UTF-8 encrypt = False expiry_date = expiry_days = expiry_prefix = follow_symlinks = False force = False get_continue = False gpg_command = /usr/bin/gpg gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_passphrase = guess_mime_type = True host_base = cephcloud.com host_bucket = %(bucket)s.cephcloud.com human_readable_sizes = False invalidate_default_index_on_cf = False invalidate_default_index_root_on_cf = True invalidate_on_cf = False kms_key = limitrate = 0 list_md5 = False log_target_prefix = long_listing = False max_delete = -1 mime_type = multipart_chunk_size_mb = 15 multipart_max_chunks = 10000 preserve_attrs = True progress_meter = True proxy_host = proxy_port = 0 put_continue = False recursive = False recv_chunk = 4096 reduced_redundancy = False requester_pays = False restore_days = 1 secret_key = Y1P8tZWsrul1ZOTMPqCiZqNMh13a1IGRxtgYC14f send_chunk = 4096 server_side_encryption = False signature_v2 = False simpledb_host = sdb.amazonaws.com skip_existing = False socket_timeout = 300 stats = False stop_on_error = False storage_class = urlencoding_mode = normal use_https = False use_mime_magic = True verbosity = WARNING website_endpoint = http://%(bucket)s.s3-website-%(location)s.amazonaws.com/ website_error = website_index = index.html
测试S3 client：
1.创建bucket：
$ s3cmd mb s3://first_bucket
Bucket 's3://first_bucket/' created2.列举buckets：
$ s3cmd -v ls
2015-09-14 06:31 s3://first_bucket3.删除bucket：
$ s3cmd rb s3://first_bucket
Bucket 's3://first_bucket/' removed4.上传文件对象：
$ s3cmd put file.txt s3://first_bucket
file.txt -> s3://first_bucket/file.txt [1 of 1] 12 of 12 100% in 0s 180.54 B/s done
在ceph管理主机上查看建立bucket的信息sudo radosgw-admin bucket list --uid=test [ "frist_bucket" ]
sudo radosgw-admin bucket stats --uid=test
[
{
"bucket": "frist_bucket",
"pool": "default.rgw.buckets.data",
"index_pool": "default.rgw.buckets.index",
"id": "89f1b54d-1a56-4cc2-a642-dab15e837719.14226.2",
"marker": "89f1b54d-1a56-4cc2-a642-dab15e837719.14226.2",
"owner": "test",
"ver": "0#3",
"master_ver": "0#0",
"mtime": "2018-01-10 10:57:24.860956",
"max_marker": "0#",
"usage": {
"rgw.main": {
"size_kb": 3,
"size_kb_actual": 4,
"num_objects": 1
}
},
"bucket_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
}
}
]
基本操作
Commands:
Make bucket
s3cmd mb s3://BUCKET
Remove bucket
s3cmd rb s3://BUCKET
List objects or buckets
s3cmd ls [s3://BUCKET[/PREFIX]]
List all object in all buckets
s3cmd la
Put file into bucket
s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]
Get file from bucket
s3cmd get s3://BUCKET/OBJECT LOCAL_FILE
Delete file from bucket
s3cmd del s3://BUCKET/OBJECT
Delete file from bucket (alias for del)
s3cmd rm s3://BUCKET/OBJECT
Restore file from Glacier storage
s3cmd restore s3://BUCKET/OBJECT
Synchronize a directory tree to S3 (checks files freshness using size and md5 checksum, unless overridden by options, see below)
s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR
Disk usage by buckets
s3cmd du [s3://BUCKET[/PREFIX]]
Get various information about Buckets or Files
s3cmd info s3://BUCKET[/OBJECT]
Copy object
s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
Modify object metadata
s3cmd modify s3://BUCKET1/OBJECT
Move object
s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
Modify Access control list for Bucket or Files
s3cmd setacl s3://BUCKET[/OBJECT]
Modify Bucket Policy
s3cmd setpolicy FILE s3://BUCKET
Delete Bucket Policy
s3cmd delpolicy s3://BUCKET
Modify Bucket CORS
s3cmd setcors FILE s3://BUCKET
Delete Bucket CORS
s3cmd delcors s3://BUCKET
Modify Bucket Requester Pays policy
s3cmd payer s3://BUCKET
Show multipart uploads
s3cmd multipart s3://BUCKET [Id]
Abort a multipart upload
s3cmd abortmp s3://BUCKET/OBJECT Id
List parts of a multipart upload
s3cmd listmp s3://BUCKET/OBJECT Id
Enable/disable bucket access logging
s3cmd accesslog s3://BUCKET
Sign arbitrary string using the secret key
s3cmd sign STRING-TO-SIGN
Sign an S3 URL to provide limited public access with expiry
s3cmd signurl s3://BUCKET/OBJECT
Fix invalid file names in a bucket
s3cmd fixbucket s3://BUCKET[/PREFIX]
Create Website from bucket
s3cmd ws-create s3://BUCKET
Delete Website
s3cmd ws-delete s3://BUCKET
Info about Website
s3cmd ws-info s3://BUCKET
Set or delete expiration rule for the bucket
s3cmd expire s3://BUCKET
Upload a lifecycle policy for the bucket
s3cmd setlifecycle FILE s3://BUCKET
Remove a lifecycle policy for the bucket
s3cmd dellifecycle s3://BUCKET
List CloudFront distribution points
s3cmd cflist
Display CloudFront distribution point parameters
s3cmd cfinfo [cf://DIST_ID]
Create CloudFront distribution point
s3cmd cfcreate s3://BUCKET
Delete CloudFront distribution point
s3cmd cfdelete cf://DIST_ID
Change CloudFront distribution point parameters
s3cmd cfmodify cf://DIST_ID
Display CloudFront invalidation request(s) status
s3cmd cfinvalinfo cf://DIST_ID[/INVAL_ID]