RHEL5.4安装pvfs-2.8.2(单节点)

1.配置环境   (root用户)

配置/etc/hosts 
202.117.10.37   amao991
//amao991作为metadata server,I/O server,client  
2.安装pvfs-2.8.2 (root用户)
(0)预备
在/root下创建 program/pvfs2-install目录,作为安装目录

soft目录(放置pvfs-2.8.2.tar.gz)

(0) 先安装bdb和mpi

db-4.3.29.tar.gz

mpich2-1.4.1p1.tar.gz

(1) 解压
[root@amao991 soft]# tar -zxvf pvfs-2.8.2.tar.gz 
(2) 编译安装

[root@amao991 pvfs-2.8.2]# ./configure -prefix=/root/program/pvfs2-install/        

-with-kernel=/usr/src/kernels/2.6.18-164.el5-i686/ 

-with-mpi=/root/program/mpich2-install/

[root@amao991 pvfs-2.8.2]# make
[root@amao991 pvfs-2.8.2]# make install
3.服务器端配置
(1)在PVFS2中，守护进程pvfs2-server担任metadata server,I/O server两个角色
(2)产生配置文件
[root@amao991 ~]# ~/program/pvfs2-install/bin/pvfs2-genconfig /etc/pvfs2-fs.conf
* Enter protocol type [Default is tcp]: 
* Enter directory name: [Default is /pvfs2-storage-space]: 
* Enter log file location [Default is /tmp/pvfs2-server.log]: 


Next you must list the hostnames of the machines that will act as
I/O servers.  Acceptable syntax is "node1, node2, ..." or "node{#-#,#,#}".
* Enter hostnames [Default is localhost]: amao991


* Would you like to verify server list (y/n) [Default is n]? 
Writing fs config file... done
//通过交互式脚本pvfs2-genconfig来生成/etc/pvfs2-fs.conf配置文件
//注意：metadata server,I/O server的机器名为amao991(不要用localhost)
(3)第一次启动前，先创建存储空间
[root@amao991 ~]# ~/program/pvfs2-install/sbin/pvfs2-server /etc/pvfs2-fs.conf -f
[root@amao991 ~]# ll /   会发现多了/pvfs2-storage-space
(4)手动启动server(默认情况下,日志写入/tmp/pvfs2-server.log)
[root@amao991 ~]# ~/program/pvfs2-install/sbin/pvfs2-server /etc/pvfs2-fs.conf 
[S 12/21 19:38] PVFS2 Server on node amao991 version 2.8.2 starting...
可以用netstat -nplt查看该服务已经启动了，占用3344端口
(5)设置开机启动(把pvfs2-server.rc放到/etc/rc.d/init.d下面，变成一个standalone服务，并设置开机就启动)
[root@amao991 ~]# cp ~/soft/pvfs-2.8.2/examples/pvfs2-server.rc /etc/rc.d/init.d/pvfs2-server
[root@amao991 ~]# chmod a+x /etc/rc.d/init.d/pvfs2-server 
[root@amao991 ~]# chkconfig pvfs2-server on   
[root@amao991 ~]# ls -al /etc/rc.d/rc3.d/S35pvfs2-server   //检查确保Linux在runlevel3纯文本启动时该服务被启动
lrwxrwxrwx 1 root root 22 Dec 22 10:10 /etc/rc.d/rc3.d/S35pvfs2-server -> ../init.d/pvfs2-server
若要即刻人工启动：
[root@amao991 ~]# /etc/rc.d/init.d/pvfs2-server start
或者[root@amao991 ~]# service pvfs2-server restart
4.配置客户端
创建挂在点，并设置开机挂在文件pvfs2tab
[root@amao991 ~]# mkdir /mnt/pvfs2
[root@amao991 ~]# touch /etc/pvfs2tab
[root@amao991 ~]# chmod a+r /etc/pvfs2tab 
[root@amao991 ~]# vi /etc/pvfs2tab
[root@amao991 ~]# cat /etc/pvfs2tab
tcp://amao991:3334/pvfs2-fs /mnt/pvfs2 pvfs2 defaults,noauto 0 0  #noauto on mount -a 
其实把上面这一行放入/etc/fstab也是可以的，就没必要创建/etc/pvfs2tab文件了
5.测试(借助pvfs2-ping, pvfs-cp, pvfs-ls命令来测试)
[root@amao991 ~]# ~/program/pvfs2-install/bin/pvfs2-ping -m /mnt/pvfs2/
(1) Parsing tab file...
(2) Initializing system interface...
(3) Initializing each file system found in tab file: /etc/pvfs2tab...
PVFS2 servers: tcp://amao991:3334
Storage name: pvfs2-fs
Local mount point: /mnt/pvfs2
/mnt/pvfs2: Ok
(4) Searching for /mnt/pvfs2/ in pvfstab...
PVFS2 servers: tcp://amao991:3334
Storage name: pvfs2-fs
Local mount point: /mnt/pvfs2


meta servers:
tcp://amao991:3334
data servers:
tcp://amao991:3334
(5) Verifying that all servers are responding...
meta servers:
tcp://amao991:3334 Ok
data servers:
tcp://amao991:3334 Ok
(6) Verifying that fsid 153623231 is acceptable to all servers...
Ok; all servers understand fs_id 153623231


(7) Verifying that root handle is owned by one server...
Root handle: 1048576
Ok; root handle is owned by exactly one server.
=============================================================
The PVFS2 filesystem at /mnt/pvfs2/ appears to be correctly configured.
[root@amao991 ~]# ~/program/pvfs2-install/bin/pvfs2-ls /mnt/pvfs2
lost+found
[root@amao991 ~]# ~/program/pvfs2-install/bin/pvfs2-cp -t /usr/lib/libc.a /mnt/pvfs2/testfile
Wrote 2817820 bytes in 0.879265 seconds. 3.056283 MB/seconds

[root@amao991 ~]# ~/program/pvfs2-install/bin/pvfs2-ls -alh /mnt/pvfs2
drwxrwxrwx    1 root     root            4.0K 2011-12-23 10:51 .
drwxrwxrwx    1 root     root            4.0K 2011-12-23 10:51 .. (faked)
-rw-r--r--    1 root     root            2.7M 2011-12-23 10:51 testfile
drwxrwxrwx    1 root     root            4.0K 2011-12-21 19:19 lost+found

[root@amao991 ~]# ~/program/pvfs2-install/bin/pvfs2-cp -t /mnt/pvfs2/testfile /tmp/testfile-out
Wrote 2817820 bytes in 0.034848 seconds. 77.114483 MB/seconds

[root@amao991 ~]# diff /tmp/testfile-out /usr/lib/libc.a

总结： 

 (1) client端

       client端通过mount将tcp://amao991:3334/pvfs2-fs 挂载在/mnt/pvfs2目录上

       client端要要用pvfs2-ls pvfs2-cp pvfs2-rm  pvfs2-mkdir等命令来访问   /mnt/pvfs2

(2)server端

      pvfs2的IO节点的存储空间是在 /pvfs2-storage-space/ 目录下 

[root@node14 ~]# ll /pvfs2-storage-space/7be3c925/
total 36
drwxr-xr-x 66 root root 4096 Mar 14 22:04 bstreams
-rw-------  1 root root 8192 Mar 14 22:04 collection_attributes.db
-rw-------  1 root root 8192 Mar 15 11:53 dataspace_attributes.db
-rw-------  1 root root    8 Mar 14 22:04 __db.001
-rw-------  1 root root 8192 Mar 15 11:53 keyval.db
drwxr-xr-x  2 root root 4096 Mar 15 11:53 stranded-bstreams

      借助：[root@node14 ~]# du /pvfs2-storage-space/7be3c925/bstreams/  

可以查看每个文件夹的大小。发现把文件放入某个文件夹了。进去看看就会明白。