一. 环境准备:
1.关闭防火墙、selinux(所有节点):
# systemctl stop firewalld ; systemctl disable firewalld
# setenforce 0
# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config //注意,网上说这种关闭selinux方法,可能导致selinux无法临时开启,可以使用以下谨慎的方法
# vim /etc/sysconfig/selinux //将SELINUX=disabled修改即可,重启虚机
# init 6
2.修改主机名(所有节点):
# hostnamectl set-hostname ceph01
# hostnamectl set-hostname ceph02
# hostnamectl set-hostname ceph03
随便找一台虚机比如ceph01上,修改hosts文件:
# vim /etc/hosts
# scp /etc/hosts 192.168.10.11:/etc/
# scp /etc/hosts 192.168.10.12:/etc/
3. SSH登录免密(ceph01控制节点):
# ssh-keygen //一路回车,生成RSA公钥
# ssh-copy-id ceph02
# ssh-copy-id ceph03
4. 配置YUM源(所有节点上):
Centos7源:
# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
epel源:
# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
新建Ceph.repo源:
# vim Ceph.repo
[ceph-nautilus] name=ceph-nautilus baseurl=http://mirrors.aliyun.com/ceph/rpm-nautilus/el7/x86_64/ enabled=1 gpgcheck=0 [ceph-nautilus-noarch] name=ceph-nautilus-noarch baseurl=http://mirrors.aliyun.com/ceph/rpm-nautilus/el7/noarch/ enabled=1 gpgcheck=0
# yum clean all
# yum makecache
5. 安装NTP服务(所有节点上):
# yum install -y chrony
在ceph01控制节点上:
# vim /etc/chrony.conf
将服务器指向了AD域服务器(网上自己找NTP服务器节点替换即可),下面是允许客户端的子网:
# systemctl restart chronyd ; systemctl enable chronyd
# chronyc sources
在ceph02、03节点上:
# vim /etc/chrony.conf
# systemctl restart chronyd ; systemctl enable chronyd
# chronyc sources
二. 部署CEPH:
1. 安装ceph和ceph-deploy:
在所有节点上:
# yum install -y ceph
在ceph01控制节点上:
# yum install -y ceph-deploy
2. 部署MON:
在ceph01控制节点上:
# mkdir ceph ; cd ceph
# ceph-deploy new ceph01 ceph02 ceph03
# vim ceph.conf
添加 overwrite_conf = true
在所有节点上:
# chown ceph:ceph -R /var/lib/ceph
回到ceph01节点(还是在/root/ceph目录下操作):
# ceph-deploy --overwrite-conf mon create-initial
在所有节点上:
# systemctl restart ceph-mon@ceph01
# systemctl restart ceph-mon@ceph02
# systemctl restart ceph-mon@ceph03
3. 部署MGR:
在ceph01上:
# ceph-deploy mgr create ceph01
# ps -ef | grep ceph
# systemctl restart ceph-mgr@ceph01
4. 部署OSD:
我这里所有节点是/dev/sdb
在ceph01节点(还是在/root/ceph目录下操作):
# ceph-deploy --overwrite-conf osd create --data /dev/sdb ceph01
# ceph-deploy --overwrite-conf osd create --data /dev/sdb ceph02
# ceph-deploy --overwrite-conf osd create --data /dev/sdb ceph03
将所有.keyring文件拷贝到所有节点/etc/ceph下:
在所有节点上重启:
# systemctl restart ceph-osd@0
# systemctl restart ceph-osd@1
# systemctl restart ceph-osd@2
在ceph01节点:
# ceph osd tree
5. 部署RGW:
这里对象存储作为单机,只安装在ceph01节点:
# yum install -y ceph-radosgw
# cepy-deploy --overwrite rgw create ceph01
# ps aux | grep radosgw
# systemctl restart ceph-radosgw@ceph01
ceph桶存储分片:
如果每个桶中对象数量较少,比如小于10000, 可以不操作此步骤, 大于10万对象,一定要设置下面的参数。
如果设计方案中,一个桶中存储对象数量大于几千万,需要关闭动态分片, 同时设置最大分片数量。
# vim /etc/ceph/ceph.conf //添加以下项
桶动态分片默认开启:
rgw_dynamic_resharding = false
桶中最大分片的数量:
rgw_override_bucket_index_max_shards = 16
# systemctl restart ceph-radosgw@ceph01 //重启服务
6. 建立S3账户:
# radosgw-admin user create --uid testid --display-name 'admin' --system
保存access_key与secret_key
7.部署Dashboard(ceph01上):
# yum install -y ceph-mgr-dashboard
# ceph mgr module enable dashboard
# ceph dashboard create-self-signed-cert
# ceph dashboard set-login-credentials admin admin //创建登录用户,并设置密码
此时,你在object gateway里面看不到bucket内容:
# ceph dashboard set-rgw-api-access-key //将access_key添加进去
# ceph dashboard set-rgw-api-secret-key //将secret_key添加进去
用网页打开https://192.168.10.10:8443
如果忘记了S3账户的KEY:
# radosgw-admin user info --uid=testid