Ceph入门到精通-创建存储桶通知

在存储桶级别创建存储桶通知。这些需要 与发送存储桶通知的目标一起发布。桶 通知是 S3 操作。

父主题：

存储桶管理

先决条件

• 运行 IBM Storage Ceph 集群，带有 Ceph Object Gateway。

• 正在运行的 HTTP 服务器、RabbitMQ 服务器或 Kafka 服务器。

• 根级访问。

• 用户访问密钥和私有密钥。

• 终结点参数。

重要：IBM 支持事件，例如 、 、 和 。IBM 还支持事件，例如 和 。ObjectCreateputpostmultipartUploadcopyObjectRemoveobject_deletes3_multi_object_delete

下面列出了创建存储桶通知的两种方法：

• 使用 boto 脚本

• 使用 AWS CLI

程序

使用 boto 脚本

1. 安装 python3-boto3 软件包：

   例

   [user@client ~]$  dnf install python3-boto3

2. 创建 S3 存储桶。

3. 创建一个 python 脚本，为 、 或 协议创建 SNS 主题：topic.pyhttpamqpkafka

   例

   import boto3
   from botocore.client import Config
   import sys
   
   # endpoint and keys from vstart
   endpoint = 'http://127.0.0.1:8000'
   access_key='0555b35654ad1656d804'
   secret_key='h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=='
   
   client = boto3.client('sns',
           endpoint_url=endpoint,
           aws_access_key_id=access_key,
           aws_secret_access_key=secret_key,
           config=Config(signature_version='s3'))
   
   attributes = {"push-endpoint": "amqp://localhost:5672", "amqp-exchange": "ex1", "amqp-ack-level": "broker"}
   
   client.create_topic(topic_name="mytopic", Attributes=attributes)

4. Run the python script for creating topic:

   Example

    python3 topic.py

5. Create a python script to create S3 bucket notification for and events:notification.pys3:objectCreates3:objectRemove

   Example

   import boto3
   import sys
   
   # bucket name as first argument
   bucketname = sys.argv[1]
   # topic ARN as second argument
   topic_arn = sys.argv[2]
   # notification id as third argument
   notification_id = sys.argv[3]
   
   # endpoint and keys from vstart
   endpoint = 'http://127.0.0.1:8000'
   access_key='0555b35654ad1656d804'
   secret_key='h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=='
   
   client = boto3.client('s3',
           endpoint_url=endpoint,
           aws_access_key_id=access_key,
           aws_secret_access_key=secret_key)
   
   # regex filter on the object name and metadata based filtering are extension to AWS S3 API
   # bucket and topic should be created beforehand
   
   topic_conf_list = [{'Id': notification_id, 
                       'TopicArn': topic_arn, 
                       'Events': ['s3:ObjectCreated:*', 's3:ObjectRemoved:*'],
                       }]
       client.put_bucket_notification_configuration(
           Bucket=bucketname,
             NotificationConfiguration={
               'TopicConfigurations': [
                 {
                   'Id': notification_name,
                   'TopicArn': topic_arn,
                   'Events': ['s3:ObjectCreated:*', 's3:ObjectRemoved:*']
                 }]})

6. Run the python script for creating the bucket notification:

   Example

    python3 notification.py

7. Create S3 objects in the bucket.

8. Fetch the notification configuration:

   Example

   endpoint = 'http://127.0.0.1:8000'
   access_key='0555b35654ad1656d804'
   secret_key='h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=='
   
   client = boto3.client('s3',
           endpoint_url=endpoint,
           aws_access_key_id=access_key,
           aws_secret_access_key=secret_key)
   
   # getting a specific notification configuration is an extension to AWS S3 API
   
   print(client.get_bucket_notification_configuration(Bucket=bucketname))

9. Optional: Delete the objects.

   1. Verify the object deletion events at the , , or receiver.httprabbitmqkafka

Using the AWS CLI

1. Create topic:

   Syntax

    aws --endpoint=_AWS_END_POINT_ sns create-topic --name NAME --attributes=ATTRIBUTES_FILE

   Example

    [user@client ~]$ aws --endpoint=http://localhost sns create-topic --name test-kafka --attributes=file://topic.json
   
    sample topic.json:
    {"push-endpoint": "kafka://localhost","verify-ssl": "False", "kafka-ack-level": "broker", "persistent":"true"}
    ref: https://docs.aws.amazon.com/cli/latest/reference/sns/create-topic.html

2. Create the bucket notification:

   Syntax

    aws s3api put-bucket-notification-configuration --bucket BUCKET_NAME --notification-configuration NOTIFICATION_FILE

   Example

    [user@client ~]$ aws s3api put-bucket-notification-configuration --bucket my-bucket --notification-configuration file://notification.json
   
    sample notification.json
    {
        "TopicConfigurations": [
            {
                "Id": "test_notification",
                "TopicArn": "arn:aws:sns:us-west-2:123456789012:test-kafka",
                "Events": [
                    "s3:ObjectCreated:*"
                ]
            }
        ]
    }

3. Fetch the notification configuration:

   Syntax

    aws s3api --endpoint=_AWS_ENDPOINT_ get-bucket-notification-configuration --bucket BUCKET_NAME

   例

    [user@client ~]$ aws s3api --endpoint=http://localhost get-bucket-notification-configuration --bucket my-bucket
    {
        "TopicConfigurations": [
            {
                "Id": "test_notification",
                "TopicArn": "arn:aws:sns:default::test-kafka",
                "Events": [
                    "s3:ObjectCreated:*"
                ]
            }
        ]
    }