hadoop-0.20.2
hbase-0.90.5
免密码ssh设置
现在确认能否不输入口令就用ssh登录localhost:
$ ssh localhost
/etc/hosts
192.168.10.147 fredmaster
如果不输入口令就无法用ssh登陆localhost,执行下面的命令:
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
HADOOP配置
1,core-site.xml
<configuration>
<!-- In: conf/core-site.xml -->
<property>
<name>hadoop.tmp.dir</name>
<value>/hadoop/hdata/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://fredmaster:9000</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri's scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri's authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>
<property>
<name>fs.checkpoint.dir</name>
<value>/hadoop/hdata/secondname</value>
<description>Determines where on the local filesystem the DFS secondary
name node should store the temporary images to merge.
If this is a comma-delimited list of directories then the image is
replicated in all of the directories for redundancy.
</description>
</property>
<property>
<name>dfs.http.address</name>
<value>fredmaster:50070</value>
<description>
The address and the base port where the dfs namenode web ui will listen on.
If the port is 0 then the server will start on a free port.
</description>
</property>
</configuration>
2,hadoop-env.sh
export JAVA_HOME=/usr/java/jdk1.6.0_29
export HADOOP_SSH_OPTS="-p 22"
3,hdfs-site.xml
<configuration>
<!-- In: conf/hdfs-site.xml -->
<property>
<name>dfs.name.dir</name>
<value>/hadoop/hdata/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/hadoop/hdata/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>3</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
<property>
<name>dfs.datanode.max.xcievers</name>
<value>4096</value>
</property>
</configuration>
4,mapred-site.xml
<configuration>
<!-- In: conf/mapred-site.xml -->
<property>
<name>mapred.job.tracker</name>
<value>fredmaster:10001</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>
</configuration>
5,masters
fredmaster
6,slaves
fredmaster
执行
格式化一个新的分布式文件系统:
$ bin/hadoop namenode -format
启动Hadoop守护进程:
$ bin/start-all.sh
hadoop dfs -mkdir /hbase
http://192.168.10.147:50070/dfshealth.jsp
浏览NameNode和JobTracker的网络接口,它们的地址默认为:
NameNode - http://localhost:50070/
JobTracker - http://localhost:50030/
完成全部操作后,停止守护进程:
$ bin/stop-all.sh
HBASE配置
1,hbase-env.sh
export HBASE_SSH_OPTS="-p 22"
export HBASE_MANAGES_ZK=true
2,hbase-site.xml
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://fredmaster:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>false</value>
<description>The mode the cluster will be in. Possible values are
false: standalone and pseudo-distributed setups with managed Zookeeper
true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh)
</description>
</property>
<property>
<name>hbase.master</name>
<value>fredmaster:60000</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>fredmaster</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/hadoop/hdata/zookeeper</value>
<description>Property from ZooKeeper's config zoo.cfg.
The directory where the snapshot is stored.
</description>
</property>
<property>
<name>zookeeper.session.timeout</name>
<value>60000</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>hbase.regionserver.restart.on.zk.expire</name>
<value>true</value>
</property>
</configuration>
3,regionservers
fredmaster
http://192.168.10.147:60010/master.jsp