centos7下openstack(rocky)和ceph（mimic）集成
1.ceph中操作创建pool（ceph mon节点操作）
ceph osd pool create volumes
ceph osd pool create images
ceph osd pool create vms
rbd pool init volumes
rbd pool init images
rbd pool init vms
ssh {your-openstack-server} sudo tee /etc/ceph/ceph.conf
2.openstack中安装包（openstack节点操作）
2.1.glance节点
yum install python-rbd -y2.2.nova和ceinder节点
yum install ceph-common -y3.配置cephx认证（ceph mon节点操作）
ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'4.为client添加key（ceph mon节点操作）
ceph auth get-or-create client.glance | ssh {your-glance-api-server} sudo tee /etc/ceph/ceph.client.glance.keyring
ssh {your-glance-api-server} sudo chown glance:glance /etc/ceph/ceph.client.glance.keyring
ceph auth get-or-create client.cinder | ssh {your-volume-server} sudo tee /etc/ceph/ceph.client.cinder.keyring
ssh {your-cinder-volume-server} sudo chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring
ceph auth get-or-create client.cinder-backup | ssh {your-cinder-backup-server} sudo tee /etc/ceph/ceph.client.cinder-backup.keyring
ssh {your-cinder-backup-server} sudo chown cinder:cinder /etc/ceph/ceph.client.cinder-backup.keyring
ceph auth get-or-create client.cinder | ssh {your-nova-compute-server} sudo tee /etc/ceph/ceph.client.cinder.keyring
ceph auth get-key client.cinder | ssh {your-compute-node} tee client.cinder.key5.计算节点libvirt配置（compute节点操作）
uuidgen
9873f259-8c5e-468a-a192-d9cd9c8921d1 #只生成一次，其他节点都使用这个uuid
cat > secret.xml <
9873f259-8c5e-468a-a192-d9cd9c8921d1
client.cinder secret
EOF
virsh secret-define --file secret.xml
virsh secret-set-value --secret 9873f259-8c5e-468a-a192-d9cd9c8921d1 --base64 $(cat client.cinder.key) && rm client.cinder.key secret.xml /etc/ceph/ceph.conf #增加下面内容
[client]
rbd cache = true
rbd cache writethrough until flush = true
admin socket = /var/run/ceph/guests/$cluster-$type.$id.$pid.$cctid.asok
log file = /var/log/qemu/qemu-guest-$pid.log
rbd concurrent management ops = 20建立相关目录并设置所有者所属组
mkdir -p /var/run/ceph/guests/ /var/log/qemu/
chown qemu:qemu /var/run/ceph/guests /var/log/qemu/6.glance配置（controller节点）
/etc/glance/glance-api.conf
[glance_store]
#stores = file,http
#default_store = file
#filesystem_store_datadir = /home/data/openstack/images
stores = rbd
default_store = rbd
rbd_store_pool = images
rbd_store_user = glance
rbd_store_ceph_conf = /etc/ceph/ceph.conf
rbd_store_chunk_size = 8重启glance服务
systemctl restart openstack-glance-api.service openstack-glance-registry.service7.cinder配置（controller节点）
/etc/cinder/cinder.conf
[DEFAULT]
#...
enabled_backends = ceph #这里可以逗号隔开写多个，每个后端对应下面一个section
glance_api_version = 2
[ceph]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph
rbd_pool = volumes
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot = false
rbd_max_clone_depth = 5
rbd_store_chunk_size = 4
rados_connect_timeout = -1
rbd_user = cinder
rbd_secret_uuid = 9873f259-8c5e-468a-a192-d9cd9c8921d1 #与计算节点libvirt步骤使用的uuid一致重启cinder-volume
systemctl restart openstack-cinder-volume.service 8.nova配置（compute节点）
/etc/nova/nova.conf
[libvirt]
rbd_user = cinder
rbd_secret_uuid = 9873f259-8c5e-468a-a192-d9cd9c8921d1
images_type = rbd
images_rbd_pool = vms
images_rbd_ceph_conf = /etc/ceph/ceph.conf
virt_type = qemu
disk_cachemodes="network=writeback"
inject_password = false
inject_key = false
inject_partition = -2
live_migration_flag="VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST"重启nova-compute服务
systemctl restart openstack-nova-compute.service 9.排错
9.1 failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/guests/ceph-client.cinder.24026.94668103843840.asok': (13) Permission denied
确保/var/run/ceph/guests目录的所有者所属组是qemu:qemu
9.2 librados: client.cinder authentication error (1) Operation not permitted
使用ceph -s --user glance/cinder #glance出的报错就用glance，cinder和nova出的报错就用cinder
如果是计算节点，使用上面命令未出现报错，则排查secert-key
virsh secret-list #查看key是否正确，如果权限问题不能解决，可以使用下面命令删掉key重做第4步最后一条命令和第5步
virsh secret-undefine {secert-key id}如果使用ceph -s --user glance/cinder 也出现权限相关的报错，去ceph mon节点进行权限排查，执行下面命令
ceph auth list
...
client.cinder
key: AQCC89VevNQAOxAA5CnLyqux18RYQf5WVS+tBw==
caps: [mon] allow r
caps: [osd] allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images
client.glance
key: AQDbs9RehroDIhAAChp5ZZR/tfLEcbwuYHRcMg==
caps: [mon] allow r
caps: [osd] allow class-read object_prefix rbd_children, allow rwx pool=images
...结果中cinder和glance的权限和上面结果一致则没有问题，如果有问题，使用下面命令删除用户，在执行3，4，5步重新做授权，uuid不要改变，这样nova和cinder配置文件不用做更改。
ceph auth rm client.cinder/client.glance
参考链接：https://docs.ceph.com/docs/master/rbd/rbd-openstack/