这篇文章适合初学者读,做下实验来之后对glusterfs有一个初步的了解,我的所有操作都是在一台机器上做的。还有一些环境配置我列举了一下,并没有很详细的列出来,相信大家应该没问题。一些第三方测试工具需要大家自行在网上下载。楼主也是一个一个下载下来的.此次下载的是3.47的版本,因为我们公司用的就是这个版本,之前试过其它版本,老是会出现服务挂起的现象,没有办法,版本回退

glusterfs  rpm包下载地址:

https://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.7/CentOS/epel-6.5/x86_64/

给大家这篇文章的一个概览,如下

环境配置2

1. rpm包安装glusterfs以及第三方工具2

1:配置本地yum2

1)创建存放rpm包的目录2

2)将下载的包上传到对应的服务器目录中2

3)安装createrepo工具并创建repo2

4) 编辑.repo文件3

5) 查看配置情况3

6)网络yum源配置(了解)4

2yum安装glusterfs5

1)安装5

2)启动9

3yum安装第三方工具9

1)安装atop fio iozone iperf sysstat9

2)安装postmark9

4:创建一个volume玩玩9

1)创建一个testvol9

2)查看testvol启动前信息10

3)查看testvol启动后信息10

4)挂载testvol并查看挂载情况10

5) 测试挂载能否正常使用10

6)卸载rpm安装的glusterfs11

2. 源码包安装glusterfs11

1:准备工作12

1)下载源码包12

2)解压源码包12

3)安装依赖包12

4)编译安装12

2:测试glusterfs是否安装成功12

1)将命令链接到path路径下12

2)启动glusterd12

3)查看volume状态12

4)尝试挂载volume13

5)查看挂载状况13

 

环境介绍

主机名 ip  角色  系统版本 软件版本

主机名

ip

系统版本

glusterfs版本

gluster-node1

192.168.11.139

centos6.5

3.4.7

glusterfs  rpm包下载地址:

https://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.7/CentOS/epel-6.5/x86_64/

环境配置

初始环境配置就不再赘述,相信大家都已经很熟练了吧

1:修改主机名

2:关闭selinux

3:设置时间同步

4:设置hosts文件

1.rpm包安装glusterfs以及第三方工具

1:配置本地yum 

1)创建存放rpm包的目录

[root@gluster-node1 ~]# mkdir -p /soft/glusterf{s,stools}

2)将下载的包上传到对应的服务器目录中

[root@gluster-node1 glusterfs]# pwd

/soft/glusterfs

[root@gluster-node1 glusterfs]# ls

glusterfs-3.4.7-1.el6.x86_64.rpm

glusterfs-api-3.4.7-1.el6.x86_64.rpm

glusterfs-api-devel-3.4.7-1.el6.x86_64.rpm

glusterfs-cli-3.4.7-1.el6.x86_64.rpm

glusterfs-debuginfo-3.4.7-1.el6.x86_64.rpm

glusterfs-devel-3.4.7-1.el6.x86_64.rpm

glusterfs-fuse-3.4.7-1.el6.x86_64.rpm

glusterfs-geo-replication-3.4.7-1.el6.x86_64.rpm

glusterfs-libs-3.4.7-1.el6.x86_64.rpm

glusterfs-rdma-3.4.7-1.el6.x86_64.rpm

glusterfs-server-3.4.7-1.el6.x86_64.rpm

[root@gluster-node1 glusterfstools]# pwd

/soft/glusterfstools

[root@gluster-node1 glusterfstools]# ls

atop-1.27-3.x86_64.rpm            iperf-2.0.4-1.el6.rf.x86_64.rpm

fio-2.1.7-1.el6.rf.x86_64.rpm     postmark_1.51.orig.tar.gz

iozone-3.394-1.el6.rf.x86_64.rpm

3)安装createrepo工具并创建repo

[root@gluster-node1 glusterfstools]# yum -y install createrepo

[root@gluster-node1 ~]# createrepo /soft/glusterfs

Spawning worker 0 with 11 pkgs

Workers Finished

Gathering worker results

 

Saving Primary metadata

Saving file lists metadata

Saving other metadata

Generating sqlite DBs

Sqlite DBs complete

[root@gluster-node1 ~]# createrepo /soft/glusterfstools/

Spawning worker 0 with 4 pkgs

Workers Finished

Gathering worker results

 

Saving Primary metadata

Saving file lists metadata

Saving other metadata

Generating sqlite DBs

Sqlite DBs complete

4)编辑.repo文件

[root@gluster-node1 ~]# vim /etc/yum.repos.d/glusterfs.repo

[glusterfs]

name=glusterfs rpm

baseurl=file:///soft/glusterfs/

enabled=1

gpgcheck=0

[root@gluster-node1 ~]# vim /etc/yum.repos.d/glusterfstools.repo

[glusterfstools]

name=glusterfs test tools

baseurl=file:///soft/glusterfstools/

enabled=1

gpgcheck=0

5)查看配置情况

[root@gluster-node1 yum.repos.d]# yum clean all

Loaded plugins: fastestmirror, refresh-packagekit, security

Cleaning repos: base extras glusterfs glusterfstools updates

Cleaning up Everything

Cleaning up list of fastest mirrors

[root@gluster-node1 yum.repos.d]# yum repolist

Loaded plugins: fastestmirror, refresh-packagekit, security

Determining fastest mirrors

 * base: mirrors.btte.net

 * extras: mirrors.btte.net

 * updates: mirrors.tuna.tsinghua.edu.cn

base                                                     | 3.7 kB     00:00     

base/primary_db                                          | 4.7 MB     00:00     

extras                                                   | 3.4 kB     00:00     

extras/primary_db                                        |  37 kB     00:00     

glusterfs                                                | 2.9 kB     00:00 ...

glusterfs/primary_db                                     | 8.6 kB     00:00 ...

glusterfstools                                           | 2.9 kB     00:00 ...

glusterfstools/primary_db                                | 5.4 kB     00:00 ...

updates                                                  | 3.4 kB     00:00     

updates/primary_db                                       | 3.7 MB     00:00     

repo id                           repo name                               status

base                              CentOS-6 - Base                         6,696

extras                            CentOS-6 - Extras                          62

glusterfs                         glusterfs rpm                              11

glusterfstools                    glusterfs test tools                        4

updates                           CentOS-6 - Updates                        686

repolist: 7,459

6)网络yum源配置(了解)

[root@gluster-node1 ~]# cat /etc/yum.repos.d/gluster.repo

[glusterfs]

name=glusterfs

baseurl=http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.7/CentOS/epel-6.5/x86_64/

enabled=1

gpgcheck=0

[root@gluster-node1 ~]# yum clean all

[root@gluster-node1 ~]# yum repolist

repo id                            repo name                                    status

base                               CentOS-6 - Base                              6,696

extras                             CentOS-6 - Extras                               62

glusterfs                          glusterfs                                       11

updates                            CentOS-6 - Updates                             686

repolist: 7,455

2yum安装glusterfs

1)安装

[root@gluster-node1 glusterfs]# pwd

/soft/glusterfs

[root@gluster-node1 glusterfs]# yum -y install glusterfs-*

Loaded plugins: fastestmirror, refresh-packagekit, security

Bad id for repo: root@gluster-node1 ~, byte = @ 4

Loading mirror speeds from cached hostfile

 * base: mirrors.btte.net

 * extras: mirrors.tuna.tsinghua.edu.cn

 * updates: mirrors.tuna.tsinghua.edu.cn

Setting up Install Process

Examining glusterfs-3.4.7-1.el6.x86_64.rpm: glusterfs-3.4.7-1.el6.x86_64

Marking glusterfs-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-api-3.4.7-1.el6.x86_64.rpm: glusterfs-api-3.4.7-1.el6.x86_64

Marking glusterfs-api-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-api-devel-3.4.7-1.el6.x86_64.rpm: glusterfs-api-devel-3.4.7-1.el6.x86_64

Marking glusterfs-api-devel-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-cli-3.4.7-1.el6.x86_64.rpm: glusterfs-cli-3.4.7-1.el6.x86_64

Marking glusterfs-cli-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-debuginfo-3.4.7-1.el6.x86_64.rpm: glusterfs-debuginfo-3.4.7-1.el6.x86_64

Marking glusterfs-debuginfo-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-devel-3.4.7-1.el6.x86_64.rpm: glusterfs-devel-3.4.7-1.el6.x86_64

Marking glusterfs-devel-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-fuse-3.4.7-1.el6.x86_64.rpm: glusterfs-fuse-3.4.7-1.el6.x86_64

Marking glusterfs-fuse-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-geo-replication-3.4.7-1.el6.x86_64.rpm: glusterfs-geo-replication-3.4.7-1.el6.x86_64

Marking glusterfs-geo-replication-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-libs-3.4.7-1.el6.x86_64.rpm: glusterfs-libs-3.4.7-1.el6.x86_64

Marking glusterfs-libs-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-rdma-3.4.7-1.el6.x86_64.rpm: glusterfs-rdma-3.4.7-1.el6.x86_64

Marking glusterfs-rdma-3.4.7-1.el6.x86_64.rpm to be installed

Examining glusterfs-server-3.4.7-1.el6.x86_64.rpm: glusterfs-server-3.4.7-1.el6.x86_64

Marking glusterfs-server-3.4.7-1.el6.x86_64.rpm to be installed

Resolving Dependencies

--> Running transaction check

---> Package glusterfs.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-api.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-api-devel.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-cli.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-debuginfo.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-devel.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-fuse.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-geo-replication.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-libs.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-rdma.x86_64 0:3.4.7-1.el6 will be installed

---> Package glusterfs-server.x86_64 0:3.4.7-1.el6 will be installed

--> Processing Dependency: xfsprogs for package: glusterfs-server-3.4.7-1.el6.x86_64

--> Running transaction check

---> Package xfsprogs.x86_64 0:3.1.1-19.el6 will be installed

--> Finished Dependency Resolution

 

Dependencies Resolved

 

==================================================================================

 Package         Arch   Version     Repository                               Size

==================================================================================

Installing:

 glusterfs       x86_64 3.4.7-1.el6 /glusterfs-3.4.7-1.el6.x86_64           3.6 M

 glusterfs-api   x86_64 3.4.7-1.el6 /glusterfs-api-3.4.7-1.el6.x86_64       126 k

 glusterfs-api-devel

                 x86_64 3.4.7-1.el6 /glusterfs-api-devel-3.4.7-1.el6.x86_64  20 k

 glusterfs-cli   x86_64 3.4.7-1.el6 /glusterfs-cli-3.4.7-1.el6.x86_64       245 k

 glusterfs-debuginfo

                 x86_64 3.4.7-1.el6 /glusterfs-debuginfo-3.4.7-1.el6.x86_64  27 M

 glusterfs-devel x86_64 3.4.7-1.el6 /glusterfs-devel-3.4.7-1.el6.x86_64     450 k

 glusterfs-fuse  x86_64 3.4.7-1.el6 /glusterfs-fuse-3.4.7-1.el6.x86_64      205 k

 glusterfs-geo-replication

                 x86_64 3.4.7-1.el6 /glusterfs-geo-replication-3.4.7-1.el6.x86_64

                                                                            472 k

 glusterfs-libs  x86_64 3.4.7-1.el6 /glusterfs-libs-3.4.7-1.el6.x86_64      669 k

 glusterfs-rdma  x86_64 3.4.7-1.el6 /glusterfs-rdma-3.4.7-1.el6.x86_64       81 k

 glusterfs-server

                 x86_64 3.4.7-1.el6 /glusterfs-server-3.4.7-1.el6.x86_64    1.5 M

Installing for dependencies:

 xfsprogs        x86_64 3.1.1-19.el6

                                    base                                    725 k

 

Transaction Summary

==================================================================================

Install      12 Package(s)

 

Total size: 35 M

Total download size: 725 k

Installed size: 37 M

Downloading Packages:

xfsprogs-3.1.1-19.el6.x86_64.rpm                           | 725 kB     00:00     

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

  Installing : glusterfs-libs-3.4.7-1.el6.x86_64                             1/12

  Installing : glusterfs-3.4.7-1.el6.x86_64                                  2/12

  Installing : glusterfs-fuse-3.4.7-1.el6.x86_64                             3/12

  Installing : glusterfs-devel-3.4.7-1.el6.x86_64                            4/12

  Installing : glusterfs-api-3.4.7-1.el6.x86_64                              5/12

  Installing : glusterfs-cli-3.4.7-1.el6.x86_64                              6/12

  Installing : xfsprogs-3.1.1-19.el6.x86_64                                  7/12

  Installing : glusterfs-server-3.4.7-1.el6.x86_64                           8/12

  Installing : glusterfs-geo-replication-3.4.7-1.el6.x86_64                  9/12

  Installing : glusterfs-api-devel-3.4.7-1.el6.x86_64                       10/12

  Installing : glusterfs-rdma-3.4.7-1.el6.x86_64                            11/12

  Installing : glusterfs-debuginfo-3.4.7-1.el6.x86_64                       12/12

  Verifying  : glusterfs-fuse-3.4.7-1.el6.x86_64                             1/12

  Verifying  : glusterfs-3.4.7-1.el6.x86_64                                  2/12

  Verifying  : glusterfs-server-3.4.7-1.el6.x86_64                           3/12

  Verifying  : glusterfs-libs-3.4.7-1.el6.x86_64                             4/12

  Verifying  : glusterfs-devel-3.4.7-1.el6.x86_64                            5/12

  Verifying  : glusterfs-rdma-3.4.7-1.el6.x86_64                             6/12

  Verifying  : glusterfs-geo-replication-3.4.7-1.el6.x86_64                  7/12

  Verifying  : glusterfs-debuginfo-3.4.7-1.el6.x86_64                        8/12

  Verifying  : glusterfs-api-devel-3.4.7-1.el6.x86_64                        9/12

  Verifying  : glusterfs-api-3.4.7-1.el6.x86_64                             10/12

  Verifying  : xfsprogs-3.1.1-19.el6.x86_64                                 11/12

  Verifying  : glusterfs-cli-3.4.7-1.el6.x86_64                             12/12

 

Installed:

  glusterfs.x86_64 0:3.4.7-1.el6                                                  

  glusterfs-api.x86_64 0:3.4.7-1.el6                                              

  glusterfs-api-devel.x86_64 0:3.4.7-1.el6                                        

  glusterfs-cli.x86_64 0:3.4.7-1.el6                                              

  glusterfs-debuginfo.x86_64 0:3.4.7-1.el6                                        

  glusterfs-devel.x86_64 0:3.4.7-1.el6                                            

  glusterfs-fuse.x86_64 0:3.4.7-1.el6                                             

  glusterfs-geo-replication.x86_64 0:3.4.7-1.el6                                  

  glusterfs-libs.x86_64 0:3.4.7-1.el6                                             

  glusterfs-rdma.x86_64 0:3.4.7-1.el6                                             

  glusterfs-server.x86_64 0:3.4.7-1.el6                                           

 

Dependency Installed:

  xfsprogs.x86_64 0:3.1.1-19.el6                                                  

 

Complete!

2)启动

[root@gluster-node1 glusterfs]# service glusterd start

Starting glusterd:                                         [确定]

3yum安装第三方工具

1)安装atop fio iozone iperf sysstat

[root@gluster-node1 glusterfs]# yum -y install atop fio iozone iperf sysstat

2)安装postmark

postmark在网上找不到rpm包,下载了个c文件

[root@gluster-node1 glusterfstools]# pwd

/soft/glusterfstools

[root@gluster-node1 glusterfstools]# tar -zxf postmark_1.51.orig.tar.gz

[root@gluster-node1 glusterfstools]# cd postmark-1.51/

[root@gluster-node1 postmark-1.51]# ls

postmark-1.51.c

[root@gluster-node1 postmark-1.51]# gcc postmark-1.51.c -o /usr/bin/postmark

/tmp/ccU1fr6U.o: In function `cli_show':

postmark-1.51.c:(.text+0x26f9): warning: the `getwd' function is dangerous and should not be used.

4:创建一个volume玩玩

这个测试是找一台单机服务器作为测试,所有的操作都是在这台服务器上进行

1)创建一个testvol

[root@gluster-node1 postmark-1.51]# gluster volume create testvol 192.168.11.139:/data/brick5 force

volume create: testvol: success: please start the volume to access data

2)查看testvol启动前信息

[root@gluster-node1 postmark-1.51]# gluster volume status

Volume testvol is not started

[root@gluster-node1 postmark-1.51]# gluster volume info

Volume Name: testvol

Type: Distribute

Volume ID: dd9da9ff-026e-4c1a-a0a1-720a7df2b7f3

Status: Created

Number of Bricks: 1

Transport-type: tcp

Bricks:

Brick1: 192.168.11.139:/data/brick5

3)查看testvol启动后信息

[root@gluster-node1 postmark-1.51]# gluster volume start testvol

volume start: testvol: success

[root@gluster-node1 postmark-1.51]# gluster volume info

Volume Name: testvol

Type: Distribute

Volume ID: dd9da9ff-026e-4c1a-a0a1-720a7df2b7f3

Status: Started

Number of Bricks: 1

Transport-type: tcp

Bricks:

Brick1: 192.168.11.139:/data/brick5

4)挂载testvol并查看挂载情况

[root@gluster-node1 postmark-1.51]# mount -t glusterfs 192.168.11.139:/testvol /mnt

[root@gluster-node1 postmark-1.51]# df -h

Filesystem                    Size  Used Avail Use% Mounted on

/dev/mapper/VolGroup-lv_root   18G  3.8G   14G  23% /

tmpfs                         364M  228K  364M   1% /dev/shm

/dev/sda1                     485M   35M  426M   8% /boot

192.168.11.139:/testvol        18G  3.8G   14G  23% /mnt

5)测试挂载能否正常使用

[root@gluster-node1 postmark-1.51]# cd /mnt/

[root@gluster-node1 mnt]# ls

[root@gluster-node1 mnt]# touch a

[root@gluster-node1 mnt]# vim a

[root@gluster-node1 mnt]# cat a

this is a demo

hello world

[root@gluster-node1 mnt]# df -h

Filesystem                    Size  Used Avail Use% Mounted on

/dev/mapper/VolGroup-lv_root   18G  3.8G   14G  23% /

tmpfs                         364M  228K  364M   1% /dev/shm

/dev/sda1                     485M   35M  426M   8% /boot

192.168.11.139:/testvol        18G  3.8G   14G  23% /mnt

[root@gluster-node1 mnt]#  cd /data/brick5/

[root@gluster-node1 brick5]# ls

a

[root@gluster-node1 brick5]# cat a

this is a demo

hello world

6)卸载rpm安装的glusterfs

[root@gluster-node1 brick5]# service glusterd stop

[root@gluster-node1 brick5]# rpm -qa | grep glusterfs      [确定]

glusterfs-api-3.4.7-1.el6.x86_64

glusterfs-geo-replication-3.4.7-1.el6.x86_64

glusterfs-3.4.7-1.el6.x86_64

glusterfs-fuse-3.4.7-1.el6.x86_64

glusterfs-debuginfo-3.4.7-1.el6.x86_64

glusterfs-devel-3.4.7-1.el6.x86_64

glusterfs-server-3.4.7-1.el6.x86_64

glusterfs-rdma-3.4.7-1.el6.x86_64

glusterfs-libs-3.4.7-1.el6.x86_64

glusterfs-cli-3.4.7-1.el6.x86_64

glusterfs-api-devel-3.4.7-1.el6.x86_64

[root@gluster-node1 brick5]# rpm -qa | grep glusterfs | xargs rpm -e

[root@gluster-node1 brick5]# rpm -qa |grep glusterfs

2.源码包安装glusterfs

1:准备工作

1)下载源码包

[root@gluster-node1 soft]# wget https://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.7/glusterfs-3.4.7.tar.gz

2)解压源码包

[root@gluster-node1 soft]# ls

glusterfs  glusterfs-3.4.7.tar.gz  glusterfstools

[root@gluster-node1 soft]# tar -zxf glusterfs-3.4.7.tar.gz

[root@gluster-node1 soft]# ls

glusterfs  glusterfs-3.4.7  glusterfs-3.4.7.tar.gz  glusterfstools

3)安装依赖包

[root@gluster-node1 glusterfs-3.4.7]# yum -y install flex bison openssl-devel

[root@gluster-node1 soft]# cd glusterfs-3.4.7

4)编译安装

[root@gluster-node1 glusterfs-3.4.7]# ./configure --prefix=/usr/local/glusterfs

[root@gluster-node1 glusterfs-3.4.7]# make

[root@gluster-node1 glusterfs-3.4.7]# make install

2:测试glusterfs是否安装成功

1)将命令链接到path路径下

[root@gluster-node1 ~]# ln -s /usr/local/glusterfs/sbin/gluster /usr/sbin/gluster

2)启动glusterd

[root@gluster-node1 ~]# service glusterd status

glusterd 已死,但是 subsys 被锁

[root@gluster-node1 ~]# service glusterd start

Starting glusterd:                                         [确定]

[root@gluster-node1 ~]# service glusterd status

glusterd (pid  81862) 正在运行...

3)查看volume状态

[root@gluster-node1 ~]# gluster volume info

Volume Name: testvol

Type: Distribute

Volume ID: dd9da9ff-026e-4c1a-a0a1-720a7df2b7f3

Status: Started

Number of Bricks: 1

Transport-type: tcp

Bricks:

Brick1: 192.168.11.139:/data/brick5

4)尝试挂载volume

[root@gluster-node1 ~]# mount -t glusterfs 192.168.0.139:/testvol /mnt

/sbin/mount.glusterfs: according to mtab, GlusterFS is already mounted on /mnt

5)查看挂载状况

[root@gluster-node1 ~]# df -h

Filesystem                    Size  Used Avail Use% Mounted on

/dev/mapper/VolGroup-lv_root   18G  3.9G   13G  24% /

tmpfs                         364M  228K  364M   1% /dev/shm

/dev/sda1                     485M   35M  426M   8% /boot

192.168.11.139:/testvol        18G  3.9G   13G  24% /mnt