Ceph对象存储快速入门

安装ceph rgw

在ceph admin节点执行下述命令

[root@ceph-admin ~]# ceph-deploy install --rgw  ceph-admin

创建对象存储网关实例

[root@ceph-admin ceph-cluster]# ceph-deploy rgw create ceph-admin
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (2.0.1): /usr/bin/ceph-deploy rgw create ceph-admin
[ceph_deploy.cli][INFO  ] ceph-deploy options:
[ceph_deploy.cli][INFO  ]  username                      : None
[ceph_deploy.cli][INFO  ]  verbose                       : False
[ceph_deploy.cli][INFO  ]  rgw                           : [('ceph-admin', 'rgw.ceph-admin')]
[ceph_deploy.cli][INFO  ]  overwrite_conf                : False
[ceph_deploy.cli][INFO  ]  subcommand                    : create
[ceph_deploy.cli][INFO  ]  quiet                         : False
[ceph_deploy.cli][INFO  ]  cd_conf                       : 
[ceph_deploy.cli][INFO  ]  cluster                       : ceph
[ceph_deploy.cli][INFO  ]  func                          : 
[ceph_deploy.cli][INFO  ]  ceph_conf                     : None
[ceph_deploy.cli][INFO  ]  default_release               : False
[ceph_deploy.rgw][DEBUG ] Deploying rgw, cluster ceph hosts ceph-admin:rgw.ceph-admin
[ceph-admin][DEBUG ] connected to host: ceph-admin 
[ceph-admin][DEBUG ] detect platform information from remote host
[ceph-admin][DEBUG ] detect machine type
[ceph_deploy.rgw][INFO  ] Distro info: CentOS Linux 7.8.2003 Core
[ceph_deploy.rgw][DEBUG ] remote host will use systemd
[ceph_deploy.rgw][DEBUG ] deploying rgw bootstrap to ceph-admin
[ceph-admin][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph-admin][WARNIN] rgw keyring does not exist yet, creating one
[ceph-admin][DEBUG ] create a keyring file
[ceph-admin][DEBUG ] create path recursively if it doesn't exist
[ceph-admin][INFO  ] Running command: ceph --cluster ceph --name client.bootstrap-rgw --keyring /var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create client.rgw.ceph-admin osd allow rwx mon allow rw -o /var/lib/ceph/radosgw/ceph-rgw.ceph-admin/keyring
[ceph-admin][INFO  ] Running command: systemctl enable [email protected]
[ceph-admin][WARNIN] Created symlink from /etc/systemd/system/ceph-radosgw.target.wants/[email protected] to /usr/lib/systemd/system/[email protected].
[ceph-admin][INFO  ] Running command: systemctl start [email protected]
[ceph-admin][INFO  ] Running command: systemctl enable ceph.target
[ceph_deploy.rgw][INFO  ] The Ceph Object Gateway (RGW) is now running on host ceph-admin and default port 7480

默认启用7480端口，若是要更改启用的端口，可以通过修改配置文件，添加以下参数，并重启ceph-radosgw.service

在 ceph.conf [global]段落后面添加，注意这里client-node是节点名称
[client.rgw.client-node]
rgw_frontends = "civetweb port=80"
例如：节点名称为ceph-admin
[client.rgw.ceph-admin]
rgw_frontends = "civetweb port=80"
更新配置文件
[root@ceph-admin ceph-cluster]# ceph-deploy --overwrite-conf config push ceph-admin
重启ceph-radosgw
[root@ceph-admin ceph-cluster]# systemctl restart [email protected]

访问节点80端口能看到如下信息

anonymous

gateway的使用

要使用REST接口，首先为S3接口创建一个初始Ceph对象网关用户。然后，为Swift创建一个子用户。然后验证创建的用户是否能够访问网关。

创建testuser用户
[root@ceph-admin ceph-cluster]#  radosgw-admin user create --uid="testuser" --display-name="First User"
{
    "user_id": "testuser",
    "display_name": "First User",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [],
    "keys": [
        {
            "user": "testuser",
            "access_key": "63C4QLTMEHB8PZF9EQCY",
            "secret_key": "8UWfMAMr6sX7o7lLtdLZMKu07uuP2VlXtZblaXfX"
        }
    ],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

创建swift用户

[root@ceph-admin ceph-cluster]# radosgw-admin subuser create --uid=testuser --subuser=testuser:swift --access=full
{
    "user_id": "testuser",
    "display_name": "First User",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [
        {
            "id": "testuser:swift",
            "permissions": "full-control"
        }
    ],
    "keys": [
        {
            "user": "testuser",
            "access_key": "63C4QLTMEHB8PZF9EQCY",
            "secret_key": "8UWfMAMr6sX7o7lLtdLZMKu07uuP2VlXtZblaXfX"
        }
    ],
    "swift_keys": [
        {
            "user": "testuser:swift",
            "secret_key": "Rece2VLigrD5hMwhMLXO1Bych14WeJsAIUGsFa8J"
        }
    ],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

创建密钥

[root@ceph-admin ceph-cluster]#  radosgw-admin key create --subuser=testuser:swift --key-type=swift --gen-secret
{
    "user_id": "testuser",
    "display_name": "First User",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [
        {
            "id": "testuser:swift",
            "permissions": "full-control"
        }
    ],
    "keys": [
        {
            "user": "testuser",
            "access_key": "63C4QLTMEHB8PZF9EQCY",
            "secret_key": "8UWfMAMr6sX7o7lLtdLZMKu07uuP2VlXtZblaXfX"
        }
    ],
    "swift_keys": [
        {
            "user": "testuser:swift",
            "secret_key": "Oe8c7JsAJobqS7IN9sFiNCbLDzgy8Ya1F8mr5LwN"
        }
    ],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

认证访问

通过S3

我们通过一个python脚本认证s3的访问，需要安装python-boto包

[root@ceph-admin ceph-cluster]# yum install python-boto -y
[root@ceph-admin ceph-cluster]# cat s3test.py 
import boto.s3.connection

access_key = '63C4QLTMEHB8PZF9EQCY'
secret_key = '8UWfMAMr6sX7o7lLtdLZMKu07uuP2VlXtZblaXfX'
conn = boto.connect_s3(
        aws_access_key_id=access_key,
        aws_secret_access_key=secret_key,
        host='ceph-admin', port=80,
        is_secure=False, calling_format=boto.s3.connection.OrdinaryCallingFormat(),
       )

bucket = conn.create_bucket('my-new-bucket')
for bucket in conn.get_all_buckets():
    print "{name} {created}".format(
        name=bucket.name,
        created=bucket.creation_date,
    )
    
此处的key为radosgw-admin user create命令生成的信息，如果忘记了可通过下述命令查看
radosgw-admin user info --uid testuser  
[root@ceph-admin ceph-cluster]# python s3test.py
my-new-bucket 2020-06-14T02:13:58.424Z

通过Swift

[root@ceph-admin ceph-cluster]#  yum install python-setuptools -y
[root@ceph-admin ceph-cluster]# easy_install pip
Searching for pip
Best match: pip 20.1.1
Adding pip 20.1.1 to easy-install.pth file
Installing pip script to /usr/bin
Installing pip3.8 script to /usr/bin
Installing pip3 script to /usr/bin

Using /usr/lib/python2.7/site-packages
Processing dependencies for pip
Finished processing dependencies for pip

[root@ceph-admin ceph-cluster]#  pip install --upgrade setuptools
[root@ceph-admin ceph-cluster]#  pip install --upgrade python-swiftclient
此处的-K 为 radosgw-admin key create 生成的"swift_keys"中的"secret_key"
[root@ceph-admin ceph-cluster]# swift -V 1 -A http://10.10.128.174:80/auth -U testuser:swift -K 'Oe8c7JsAJobqS7IN9sFiNCbLDzgy8Ya1F8mr5LwN' list

my-new-bucket

导读：
1.从零部署一个ceph集群
2.ceph block device与cephfs快速入门