boto3操作minio总结

aws S3 SDK 校验minio 服务

S3 SDK 的Python包名: boto3 (pip install -U boto3)

校验内容:

1. 通过access_key_id和secret_access_key连接到Minio
2. 查看所有bucket
3. 新建bucket
4. 查看一个bucket下的所有object
5. 删除bucket
6. 上传obj
7. 下载obj
8. 删除bucket下所有object
9. 删除bucket下某个object

校验代码:

import botocore

from boto3.session import Session

aws_access_key_id = 'your_access_key'
aws_secret_access_key = 'your_secret_access_key'

session = Session(aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key)

# 连接到minio
s3 = session.resource('s3', endpoint_url='http://xxx.xx.xx.xx:9000')

# 查看所有bucket
for bucket in s3.buckets.all():
    print('bucket name:%s' % bucket.name)

# 新建一个bucket(bucket name 中不能有_下划线)
s3.create_bucket(Bucket='mybucket')

# 查看一个bucket下的所有object
bucket_name = 'test1'
bucket = s3.Bucket(bucket_name)
for obj in bucket.objects.all():
    print('obj name:%s' % obj.key)

# 删除bucket(只能删除空的bucket)
try:
    bucket.delete()
except botocore.exceptions.ClientError as e:
    print('bucket is not empty')

# 上传obj
s3.Object(bucket_name, obj_name).upload_file(local_file)
# 或
# bucket.put_object(Key=obj_name, Body=open(local_file, 'rb'))

# 下载obj
s3.Object(bucket_name, obj_name).download_file(local_file)

# 删除bucket下所有object
bucket.objects.filter().delete()

# 删除bucket下某个object
bucket.objects.filter(Prefix=obj_name).delete()

一些其他方法

检查一下这个bucket是否存在

bucket_name = 'test1'
exists = True

try:
    s3.meta.client.head_bucket(Bucket=bucket_name)
except botocore.exceptions.ClientError as e:
    # 如果抛出客户端错误，则检查它是否为404错误。
    # 如果是404错误，则存储桶不存在
    error_code = int(e.response['Error']['Code'])
    if error_code == 404:
        exists = False

循环下载大文件

def download_big_file():
    obj = s3.Object(bucket_name, object_name)
    chunk_size = 64 * 1024
    stream = obj.get().get('Body')
    with open(local_fails, 'wb') as f:
        while True:
            data = stream.read(chunk_size)
            if not len(data):
                raise StopIteration
            f.write(data)

查看bucket是否存在文件

def obj_is_exist(obj_name):
    for obj in bucket.objects.filter(Prefix=obj_name):
        if obj.key == obj_name:
            return True
    return False