Glusterfs 分布式存储系统配置

在服务器集群中,为了同步更新APP内容,利用Glusterfs分布式存储功达到相同的APP只更新一次,其他机器同步更新

试验环境:Vmware Fusion3.1.1 CentOS 5.5

1.安装
#yum -y install glusterfs-cluster glusterfs-server

2.设置两台机器的别名分别为svr1,svr2

3.两台机器的服务端配置文件和客户端文件如下:
服务端配置文件glusterfsd.vol
[root@svr1 ~]# cat /etc/glusterfs/glusterfsd.vol
### file: server-volume.vol.sample

#####################################
###  GlusterFS Server Volume File  ##
#####################################

#### CONFIG FILE RULES:
### "#" is comment character.
### - Config file is case sensitive
### - Options within a volume block can be in any order.
### - Spaces or tabs are used as delimitter within a line.
### - Multiple values to options will be : delimitted.
### - Each option should end within a line.
### - Missing or commented fields will assume default values.
### - Blank/commented lines are allowed.
### - Sub-volumes should already be defined above before referring.

### Export volume "brick" with the contents of "/home/export" directory.
volume posix
  type storage/posix                   # POSIX FS translator
  option directory /opt/export        # Export this directory
end-volume

volume locks
  type features/locks
# option mandatory-locks on
  subvolumes posix
end-volume

volume brick
  type performance/io-threads
  option thread-count 8
  subvolumes locks
end-volume


### Add network serving capability to above brick.
volume server
  type protocol/server
  option transport-type tcp
# option transport-type unix
# option transport-type ib-sdp
# option transport.socket.bind-address 192.168.1.10     # Default is to listen on all interfaces
# option transport.socket.listen-port 6996              # Default is 6996

# option transport-type ib-verbs
# option transport.ib-verbs.bind-address 192.168.1.10     # Default is to listen on all interfaces
# option transport.ib-verbs.listen-port 6996              # Default is 6996
# option transport.ib-verbs.work-request-send-size  131072
# option transport.ib-verbs.work-request-send-count 64
# option transport.ib-verbs.work-request-recv-size  131072
# option transport.ib-verbs.work-request-recv-count 64

# option client-volume-filename /etc/glusterfs/glusterfs-client.vol
  subvolumes brick
# NOTE: Access to any volume through protocol/server is denied by
# default. You need to explicitly grant access through # "auth"
# option.
  option auth.addr.brick.allow 127.0.0.1,192.168.*  # Allow access to "brick" volume
end-volume

客户端配置文件glusterfs.vol
[root@svr1 ~]# cat /etc/glusterfs/glusterfs.vol
### file: client-volume.vol.sample

#####################################
###  GlusterFS Client Volume File  ##
#####################################

#### CONFIG FILE RULES:
### "#" is comment character.
### - Config file is case sensitive
### - Options within a volume block can be in any order.
### - Spaces or tabs are used as delimitter within a line.
### - Each option should end within a line.
### - Missing or commented fields will assume default values.
### - Blank/commented lines are allowed.
### - Sub-volumes should already be defined above before referring.

### Add client feature and attach to remote subvolume
volume svr1
  type protocol/client
  option transport-type tcp
  option remote-host svr1
  option remote-subvolume brick       #name of the remote volume
end-volume

volume svr2
  type protocol/client
  option transport-type tcp
  option remote-host svr2
  option remote-subvolume brick
end-volume

volume replicate1
  type cluster/replicate
  subvolumes svr1
end-volume

volume replicate2
  type cluster/replicate
  subvolumes svr2
end-volume

volume distribute
  type cluster/distribute
  subvolumes replicate1 replicate2
#  subvolumes replicate1
end-volume

volume writebehind
  type performance/write-behind
  option cache-size 1MB
  subvolumes distribute
end-volume

volume cache
  type performance/io-cache
  option cache-size 64MB
  subvolumes writebehind
end-volume

4.接下来启动服务端
[root@svr1 ~]# glusterfsd -f /etc/glusterfs/glusterfsd.vol
执行ps fax|grep gluseterfs查看是否有启动

使用以上命令操作svr2

5.启动客户端
启动客户端之前先挂起fuse
[root@svr1 ~]# modprobe fuse

启动客户端
[root@svr1 ~]# glusterfs -l /var/log/glusterfs/opt-glusterfs.log -f /etc/glusterfs/glusterfs.vol /opt/clusterfs

使用df -h 查看挂载是否成功
[root@svr1 ~]# df -h
文件系统              容量  已用 可用 已用% 挂载点
/dev/hda2             5.7G  4.6G  858M  85% /
/dev/hda3             2.9G   69M  2.7G   3% /home
/dev/hda1              99M   12M   82M  13% /boot
tmpfs                 502M     0  502M   0% /dev/shm
glusterfs#/etc/glusterfs/glusterfs.vol
                       12G  9.0G  1.9G  84% /opt/clusterfs

如果没有如上显示，则说明挂载没有成功，可以查看/var/log/glusterfs/opt-glusterfs.log下建立的日志进行排错

最后使用以上命令进行svr2的操作
注:假如重启到glusterfs挂连接不上,可以使用命令取消挂起
[root@svr1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_webserver2-lv_root
                       50G  2.0G   45G   5% /
tmpfs                 1.5G     0  1.5G   0% /dev/shm
/dev/sda1             485M   37M  423M   8% /boot
/dev/mapper/vg_webserver2-lv_home
                       46G  180M   44G   1% /home
df: `/opt/clusterfs': Transport endpoint is not connected
[root@svr1 ~]#umount /opt/clusterfs
root@svr2 ~]#umount /opt/clusterfs
================================
最后的效果是,不管在哪台机器上的/opt/glusterfs下新建更新文件,都会同步更新到另一台机器上,当你相同的APP在N台机器上时,这是多么轻松的事啊