最近因为工作的需要在接触GlusterFS,在未来的一段时间之内应该都要与之打交道,刚刚接触分布式文件系统,对很多概念和机制都不是很清楚,通过一段时间的学习有一点浅显的认识,那么就把这段时间的了解到的内容做个简单的笔记,包括GlusterFS的基本概念,以及部署GlusterFS的方法。
1.glusterfs基本概念的介绍
因为对GlusterFS的整体架构理解的过于肤浅,那么就只能对一些简单的概念进行介绍。
brick:brick是任何在可信共享池中共享的目录(Brick is basically any directory that is meant to be shared among the trusted storage pool)。(感觉翻译的烂的一逼)。另一种说法:Each server gives one of its directories (referred to as a “brick”) into the care of GlusterFS, which combines multiple bricks to create a large data repository.
Trusted Storage Pool:这是共享文件和目录的的集合,这些文件和目录基于设计好的网络协议。(is a collection of these shared files/directories, which are based on the designed protocol.)(感觉翻译太烂了,还是直接用英文吧)
Block Storage:They are devices through which the data is being moved across systems in the form of blocks.
Cluster:In Red Hat Storage, both cluster and trusted storage pool convey the same meaning of collaboration of storage servers based on a defined protocol.
Distributed File System:A file system in which data is spread over different nodes where users can access the file without knowing the actual location of the file. User doesn’t experience the feel of remote access.
FUSE:It is a loadable kernel module which allows users to create file systems above kernel without involving any of the kernel code.
glusterd:glusterd is the GlusterFS management daemon which is the backbone of file system which will be running throughout the whole time whenever the servers are in active state.
POSIX:Portable Operating System Interface (POSIX) is the family of standards defined by the IEEE as a solution to the compatibility between Unix-variants in the form of an Application Programmable Interface (API).
RAID:Redundant Array of Independent Disks (RAID) is a technology that gives increased storage reliability through redundancy.
Subvolume:A brick after being processed by least at one translator.
Translator:A translator is that piece of code which performs the basic actions initiated by the user from the mount point. It connects one or more sub volumes.
Volume:A volumes is a logical collection of bricks. All the operations are based on the different types of volumes created by the user. The volume,in turn,is mounted by the clients like a drive.
2.GlusterFS的部署方法。
操作系统:
centos6.4:
lsb_release -a
一共有三台机器:
client:
10.23.85.47 glusterA
server:
10.23.85.48 glusterB
10.23.85.49 glusterC
glusterfs源代码:http://yunpan.cn/cVHsYCHzGGe43
(提取码:710e)(代码版本为3.6.3)
在三台机器上解压文件:
unzip gluster3.6.zip
在每台机器上进行配置编译安装,注意还有一些必要的安装包,那么写一个脚本一起安装run.sh(这个脚本放在源代码的根目录下):
yum install -y aclocal autoconf aotuheader automake libtool
yum install -y automake autoconf libtool flex bison openssl-devel libxml2-devel python-devel libaio-devel libibverbs-devel librdmacm-devel readline-devel lvm2-devel glib2-devel userspace-rcu-devel libcmocka-devel
./configure --enable-debug
sudo make
sudo make install
安装好之后执行:
glusterfs --version
说明安装完成,在三台机器上执行同样的操作。
add server to trusted storage pool:即将两个存储server节点组成一个集群,本文在第一个server结点执行,在任何一个server结点执行即可。
http://www.gluster.org/community/documentation/index.php/Gluster_3.2:_Adding_Servers_to_Trusted_Sto
rage_Pool
error:connection failed.please check if gluster daemon is operational. 需要开启glusterd守护进程
当然其他的server也要运行:service glusterd start命令,开启守护进行,否则会报错:peer probe: failed: Probe returned with unknown errno 107.
查看集群结点的信息:
在server glusterB查看:
在server glusterC查看:
以/data/gluster为共享目录,创建名为img的卷,副本数为2:
启动volume并查看卷状态:
在客户端挂载卷img:
首先用df -h命令查看挂载信息:
执行将卷img挂在到客户端的/mnt挂载点上:
在部署好之后,我们尝试对文件进行操作,看看效果:
创建文件:
client端:
server端:
删除文件:
client端:
server端:
至此,GlusterFS分布式文件系统的基本概念和搭建工作完成。
3.问题总结:
1.在挂载的时候出现"Transport endpoint is not connected"的问题。
方法一:强制卸载(http://stackoverflow.com/questions/24966676/transport-endpoint-is-not-connected )
fusermount -uz /mnt
方法二:
还有一个原因就是server端的进程glusterfsd没有启动,导致无法成功挂载,启动glusterfsd进程(
glusterfsd在volume start的时候启动:gluster volume start img)。
2.在serverA和serverB之间用scp拷贝数据"permission denied"的问题。
解决方法:首先在serverA上建立id_rsa.pub(如果之前没有建的话):ssh-keygen -t rsa,然后将生成的id_rsa.pub的内容拷贝到serverB的~/.ssh/authorized_keys文件里面,搞定。
3.在执行"gluster peer probe 10.23.85.38"操作的时候出现"peer probe: failed: Probe returned with unknown errno 107"。
有几个原因:
>每台机器上的glusterd没有启动,执行"service glusterd start"即可。
>确定机器之间是否能ping通。
Author:忆之独秀
Email:[email protected]
注明出处:http://blog.csdn.net/lavorange/article/details/44902533