CentOS7离线安装GlusterFS7.9.1
CentOS 7 离线安装GlusterFS 7.9.1
环境
| 系统ip | 主机名 | 添加磁盘并格式化 | 每个磁盘挂载点 |
|---|---|---|---|
| 192.168.32.128 | servera | /dev/sdb1 | /brick/brick |
| 192.168.32.132 | serverb | /dev/sdb1 | /brick/brick |
每台机器中均添加
192.168.32.128 servera
192.168.32.132 serverb
servera:
[root@servera /]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.32.128 servera
192.168.32.132 serverb
[root@servera /]#
serverb:
[root@serverb /]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.32.128 servera
192.168.32.132 serverb
[root@serverb /]#
安装前的准备
至少2个虚拟磁盘，一个用于安装OS (sda)，一个用于服务GlusterFS存储(sdb)。这将模拟一个真实的部署，在这个部署中，您可能希望将GlusterFS存储与操作系统安装分离。
格式化磁盘
# mkfs.ext4 /dev/sdb1
# mkdir -p /brick/brick
# mount /dev/sdb1 /brick/brick
开始安装
安装参考链接：
https://blog.csdn.net/liuskyter/article/details/111595968
https://blog.csdn.net/daydayup_gzm/article/details/52748800
https://blog.csdn.net/weixin_44729138/article/details/105663849
开始安装，其中部分依赖包因为之前已安装未列出，如安装提示缺少依赖包，请自行安装。
rpm -ivh glusterfs-libs-7.9-1.el7.x86_64.rpm --nodeps --force
rpm -ivh glusterfs-7.9-1.el7.x86_64.rpm --nodeps --force
rpm -ivh glusterfs-client-xlators-7.9-1.el7.x86_64.rpm
rpm -ivh glusterfs-api-7.9-1.el7.x86_64.rpm --nodeps --force
rpm -ivh glusterfs-cli-7.9-1.el7.x86_64.rpm --nodeps --force
rpm -ivh userspace-rcu-0.10.0-3.el7.x86_64.rpm
rpm -ivh glusterfs-fuse-7.9-1.el7.x86_64.rpm --nodeps --force
rpm -ivh glusterfs-server-7.9-1.el7.x86_64.rpm
安装包链接:
链接：https://pan.baidu.com/s/18SECuBRbIcX8BRDeoUg0sg
设置gluster开机启动，所有服务器上均须执行
systemctl start glusterd.service
systemctl enable glusterd.service
systemctl status glusterd.service
启动gluster管理守护进程，所有服务器上均须执行
service glusterd start
配置防火墙
节点上的gluster进程需要能够相互通信。为了简化此设置，请在每个节点上配置防火墙以接受来自另一个节点的所有流量
iptables -I INPUT -p all -s 192.168.32.132 -j ACCEPT
iptables -I INPUT -p all -s 192.168.32.128 -j ACCEPT
配置可信池
#servera上执行
gluster peer probe serverb
peer probe: success. Host serverb port 24007 already in peer list
#serverb上执行
gluster peer probe servera
peer probe: success. Host servera port 24007 already in peer list
创建复制卷，所有服务器均须执行
在所有服务器上均须执行：
mkdir -p /brick/brick/gv0
mkdir -p /brick/brick/gv0
任意一台服务器上执行：
gluster volume create volume1 replica 2 servera:/brick/brick/gv0 serverb:/brick/brick/gv0
Replica 2 volumes are prone to split-brain. Use Arbiter or Replica 3 to avoid this. See: http://docs.gluster.org/en/latest/Administrator%20Guide/Split%20brain%20and%20ways%20to%20deal%20with%20it/.
Do you still want to continue?
(y/n) y
volume create: gv0: success: please start the volume to access data
启动该卷
gluster volume start volume1
#volume start: gv0: success
查看卷的信息
gluster volume info
################################################
Volume Name: gv0
Type: Replicate
Volume ID: 8afec598-4867-4136-8c7c-dc3301475a47
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: servera:/brick/brick/gv0
Brick2: serverb:/brick/brick/gv0
Options Reconfigured:
transport.address-family: inet
storage.fips-mode-rchecksum: on
nfs.disable: on
performance.client-io-threads: off
注意
如果该卷未显示“已启动”，/var/log/glusterfs/glusterd.log则应检查下面的文件 以便调试和诊断情况。可以在一台或所有配置的服务器上查看这些日志
创建挂载卷目录（两台服务器均需操作）
mkdir /brick1
挂载卷（两台服务器均需操作）
servera:
mount -t glusterfs servera:/volume1 /brick1
serverb:
mount -t glusterfs serverb:/volume1 /brick1
写数据测试
在任意一台服务器上执行以下命令，将/var/log/目录下的messages日志拷贝至/brick1目录下
for i in `seq -w 1 10`; do cp -rp /var/log/messages /brick1/copy-test-$i; done
观察
servera:
[root@servera glusterfs]# cd /brick1
[root@servera brick1]# ll
total 5985
-rw-------. 1 root root 612545 May 13 01:50 copy-test-01
-rw-------. 1 root root 612545 May 13 01:50 copy-test-02
-rw-------. 1 root root 612545 May 13 01:50 copy-test-03
-rw-------. 1 root root 612545 May 13 01:50 copy-test-04
-rw-------. 1 root root 612545 May 13 01:50 copy-test-05
-rw-------. 1 root root 612545 May 13 01:50 copy-test-06
-rw-------. 1 root root 612545 May 13 01:50 copy-test-07
-rw-------. 1 root root 612545 May 13 01:50 copy-test-08
-rw-------. 1 root root 612545 May 13 01:50 copy-test-09
-rw-------. 1 root root 612545 May 13 01:50 copy-test-10
[root@servera brick1]#
同时观察serverb发现，在serverb服务器上/brick1目录下也会存在一份拷贝文件
[root@serverb glusterfs]# cd /brick1/
[root@serverb brick1]# ll
total 5985
-rw-------. 1 root root 612545 May 13 01:50 copy-test-01
-rw-------. 1 root root 612545 May 13 01:50 copy-test-02
-rw-------. 1 root root 612545 May 13 01:50 copy-test-03
-rw-------. 1 root root 612545 May 13 01:50 copy-test-04
-rw-------. 1 root root 612545 May 13 01:50 copy-test-05
-rw-------. 1 root root 612545 May 13 01:50 copy-test-06
-rw-------. 1 root root 612545 May 13 01:50 copy-test-07
-rw-------. 1 root root 612545 May 13 01:50 copy-test-08
-rw-------. 1 root root 612545 May 13 01:50 copy-test-09
-rw-------. 1 root root 612545 May 13 01:50 copy-test-10
[root@serverb brick1]#
问题汇总
1、创建卷失败
GlusterFS创建volume失败的解决方法，（* or a prefix of it is already part of a volume）
rm -rf /u01/isi/glusterdata/.glusterfs/ #删除数据目录下的数据
setfattr -x trusted.glusterfs.volume-id /u01/isi/glusterdata #清除属性信息
setfattr -x trusted.gfid /u01/isi/glusterdata #清除属性信息
重新创建即可
2、常见复制卷失败
volume create: volume1: failed: Staging failed on glusterfs02. Error: The brick
glusterfs02:/u01/isi/glusterdata is being created in the root partition. It is
recommended that you don't use the system's root partition for storage backend.
Or use 'force' at the end of the command if you want to override this behavior.
是因为创建的卷占用了/分区，在命令的后面加上force即可
gluster volume create volume1 replica 2 glusterfs01:/u01/isi/glusterdata/
glusterfs02:/u01/isi/glusterdata/ force
3、卸载过程（两台服务器均操作）
step1:卸载盘和停止卷
cd
umount /brick1/ （两台服务器均操作） #卸载盘
gluster volume stop volume1（任意一台服务器操作，以118为例）#停止卷
step2:删除卷
gluster volume delete volume1 #删除卷
gluster volume list #列出集群中的所有卷
gluster volume info #查看集群中的卷信息
gluster volume status #查看集群中的卷状态
gluster volume status all #查看集群中的卷状态
step3:删除节点
gluster peer detach serverb
step4:停用服务
systemctl stop glusterd
systemctl disable glusterd
systemctl status gluster
step5:卸载服务
yum -y remove glusterfs-server centos-release-gluster #卸载服务
rm -rf /u01/isi/glusterdata /brick1 #删除数据目录和挂载卷目录