MFS安装问题

补充一些关于日志信息的说明：
一：mfsmaster每分钟进行一次检查，发现有chunkserver register或者disconnected时，会有如下日志信息提示。

每分钟检测
Jul 23 09:38:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 09:38:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 09:38:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 09:38:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 09:38:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 09:38:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 09:38:00 mfsmanager mfsmaster[14105]: total: usedspace: 0 (0 GB), totalspace: 0 (0 GB), usage: 0.00%
Jul 23 09:39:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 09:39:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 09:39:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 09:39:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 09:39:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 09:39:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 09:39:00 mfsmanager mfsmaster[14105]: total: usedspace: 0 (0 GB), totalspace: 0 (0 GB), usage: 0.00%

chunkserver register时
Jul 23 10:02:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 10:02:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 10:02:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 10:02:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 10:02:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 10:02:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 10:02:00 mfsmanager mfsmaster[14105]: total: usedspace: 0 (0 GB), totalspace: 0 (0 GB), usage: 0.00%
Jul 23 10:02:10 mfsmanager mfsmaster[14105]: chunkserver register – ip: 192.168.1.243, port: 9422, usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB)
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: server 1 (192.168.1.243): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:03:00 mfsmanager mfsmaster[14105]: total: usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%

chunkserver connection lost 时
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: server 1 (192.168.1.243): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 09:58:00 mfsmanager mfsmaster[14105]: total: usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 09:58:41 mfsmanager mfsmaster[14105]: connection with CS(192.168.1.243) lost
Jul 23 09:58:41 mfsmanager mfsmaster[14105]: chunkserver disconnected – ip: 192.168.1.243, port: 9422, usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB)
Jul 23 09:59:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 09:59:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 09:59:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 09:59:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 09:59:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 09:59:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 09:59:00 mfsmanager mfsmaster[14105]: total: usedspace: 0 (0 GB), totalspace: 0 (0 GB), usage: 0.00%

二：chunkserver如果没有指定mfshdd.cfg文件里的分区的话，错误信息如下：

Jul 23 10:25:07 mfsmanager mfschunkserver[15464]: config: using default value for option ‘HDD_CONF_FILENAME’ – ‘/usr/local/mfs/etc/m
fshdd.cfg’
Jul 23 10:25:07 mfsmanager mfschunkserver[15464]: can’t create lock file ‘/mnt/hd1/.lock’: No such file or directory
Jul 23 10:25:07 mfsmanager mfschunkserver[15464]: init: hdd space manager failed !!!

三：chunkserver的mfschunkserver.cfg文件中，如果没有将如下选项前的#符号去除的话，日志会一直显示如下：
MASTER_HOST = 192.168.1.242

Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘HDD_CONF_FILENAME’ – ‘/usr/local/mfs/etc/mfshdd.cfg’
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘CSSERV_LISTEN_HOST’ – ‘*’
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘CSSERV_LISTEN_PORT’ – ’9422′
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘CSSERV_TIMEOUT’ – ’60′
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: listen on *:9422
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘CSTOCS_TIMEOUT’ – ’60′
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘MASTER_RECONNECTION_DELAY’ – ’5′
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘MASTER_HOST’ – ‘mfsmaster’
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘MASTER_PORT’ – ’9420′
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘MASTER_TIMEOUT’ – ’60′
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: config: using default value for option ‘BACK_LOGS’ – ’50′
Jul 23 10:29:10 mfsmanager mfschunkserver[15477]: connecting …

正常情况下则出现如下信息：

Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘HDD_CONF_FILENAME’ – ‘/usr/local/mfs/etc/mfshdd.cfg’
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘CSSERV_LISTEN_HOST’ – ‘*’
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘CSSERV_LISTEN_PORT’ – ’9422′
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘CSSERV_TIMEOUT’ – ’60′
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: listen on *:9422
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘CSTOCS_TIMEOUT’ – ’60′
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘MASTER_RECONNECTION_DELAY’ – ’5′
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘MASTER_PORT’ – ’9420′
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘MASTER_TIMEOUT’ – ’60′
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: config: using default value for option ‘BACK_LOGS’ – ’50′
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: connecting …
Jul 23 10:33:01 mfsmanager mfschunkserver[15485]: connected to Master

四：mfsmaster在检测到新的chunkserver时，在日志中会显示如下信息：
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: server 1 (192.168.1.243): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:33:00 mfsmanager mfsmaster[14105]: total: usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:33:01 mfsmanager mfsmaster[14105]: chunkserver register – ip: 192.168.1.244, port: 9422, usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB)
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: server 1 (192.168.1.244): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: server 2 (192.168.1.243): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:34:00 mfsmanager mfsmaster[14105]: total: usedspace: 2050588672 (1 GB), totalspace: 22994255872 (21 GB), usage: 8.92%

Jul 23 10:50:00 mfsmanager mfsmaster[14105]: inodes: 1
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: dirnodes: 1
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: filenodes: 0
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: chunks: 0
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: chunks to delete: 0
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: chunkservers status:
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: server 1 (192.168.1.245): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: server 2 (192.168.1.244): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: server 3 (192.168.1.243): usedspace: 1025294336 (0 GB), totalspace: 11497127936 (10 GB), usage: 8.92%
Jul 23 10:50:00 mfsmanager mfsmaster[14105]: total: usedspace: 3075883008 (2 GB), totalspace: 34491383808 (32 GB), usage: 8.92%

五：FUSE下载地址如下：

http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.7.4/fuse-2.7.4.tar.gz

（超慢，时断时续的，我找了3台不同地方的服务器在下载）
也可以在这里下载 http://qwqg.mobibal.com/wp-content/uploads/2009/06/fuse-2.7.4.tar.gz

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH

对于原文安装MFS客户端需要指定pkg_config_path环境变量的另一种解决办法是，在编译fuse时，指定其prefix为–prefix=/usr即可不需要再指定pkg_config_path的环境变量，具体内容可参考http://qwqg.mobibal.com/?p=131。

六：运行mfsmount时，fuse报错如下的解决办法：
[root@mfsmanager mfs-1.5.12]# /usr/local/mfs/bin/mfsmount -h
/usr/local/mfs/bin/mfsmount: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory

编辑/etc/ld.so.conf文件，将/usr/local/lib加入。
运行ldconfig，更新缓存。

[root@mfsmanager mfs-1.5.12]# /usr/local/mfs/bin/mfsmount -v
/usr/local/mfs/bin/mfsmount: option requires an argument — v
mfsmount v. 1.5.12
usage: /usr/local/mfs/bin/mfsmount [-r][-m][-c] [-v 0..2] [-h master host] [-p master port] [-l path] [-w mount point]

r: readonly mode
m: mount metadata
c: allow using cache
v: verbose level

defaults:
h: mfsmaster
p: 9421
l: /
w: /mnt/mfs

[root@mfsmanager mfs-1.5.12]# /usr/local/mfs/bin/mfsmount -h 192.168.1.242
fuse: device not found, try ‘modprobe fuse’ first
error in fuse_mount
[root@mfsmanager mfs-1.5.12]# modprobe fuse
[root@mfsmanager mfs-1.5.12]# /usr/local/mfs/bin/mfsmount -h 192.168.1.242
[root@mfsmanager mfs-1.5.12]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 7936288 1646880 5879752 22% /
/dev/sda5 11227664 159512 10488612 2% /data
/dev/sda1 124427 11401 106602 10% /boot
tmpfs 257720 0 257720 0% /dev/shm
MFS 33682944 3003776 30679168 9% /mnt/mfs

在一个目录下建了6个linux的虚拟机，master和chunkserver都正常，但client在mount master上的共享分区时，总是报错，即使是将client的虚拟机移到另一个分区下面还是不行，用了另外一台机器上的虚拟机可以正常访问mount。

七：master报错如下：

Jul 24 14:30:00 mfsmanager mfsmaster[3077]: inodes: 137
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: dirnodes: 2
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: filenodes: 135
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: chunks: 134
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: chunks to delete: 0
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: chunkservers status:
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: server 1 (192.168.1.243): usedspace: 1812389888 (1 GB), totalspace: 11497127936 (10 GB), usage: 15.76%
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: server 2 (192.168.1.244): usedspace: 1812381696 (1 GB), totalspace: 11497127936 (10 GB), usage: 15.76%
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: server 3 (192.168.1.245): usedspace: 1812361216 (1 GB), totalspace: 11497127936 (10 GB), usage: 15.76%
Jul 24 14:30:00 mfsmanager mfsmaster[3077]: total: usedspace: 5437132800 (5 GB), totalspace: 34491383808 (32 GB), usage: 15.76%
Jul 24 14:30:39 mfsmanager mfsmaster[3077]: chunk 9 has only invalid copies (1) – please repair it manually
Jul 24 14:30:39 mfsmanager mfsmaster[3077]: chunk 9 (0000000000000009:0000000F) – invalid copy on (192.168.1.243)

Posted in 系统.