CentOS6.4下伪分布式安装Hadoop-1.2.1

一、配置hadoop

总共需要修改4个文件，"hadoop-env.sh"，"core-site.xml"、"hdfs-site.xml"和"mapred-site.xml"。

1.hadoop-env.sh

export JAVA_HOME=/usr/java/jdk1.7.0_45

2.core-site.xml

	
		hadoop.tmp.dir
		/home/luffy/Development/hadoop-${user.name}
		A base for other temporary directories.
	
	 
		fs.default.name 
		hdfs://localhost:9000 
	

 3.hdfs-site.xml

dfs.replication
1

 4.mapred-site.xml

mapred.job.tracker
localhost:9001

修改之后，设置环境变量

#set Environment
export JAVA_HOME=/usr/java/jdk1.7.0_45
export HADOOP_INSTALL=/home/luffy/Development/hadoop-1.2.1
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$HADOOP_INSTALL/bin:$PATH

二、配置ssh免密码登陆 

因为CentOS6.4自带安装ssh，所以不需要再安装，但是要配置ssh免密码登陆，需要进行设置

1.ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

2.sudo gedit /etc/selinux/config，并修改"SELINUX=disabled"，然后source /etc/selinux/config

3.chmod 600 ~/.ssh/authorized_keys

chmod 700 ~/.ssh

4.至此，设置已完成，注销再登陆即可。

 三、hadoop测试

1.hadoop namenode -format

2.start-all.sh

3.此时，能访问到http://localhost:50070/和http://localhost:50030/

4.cd /home/luffy/Development/hadoop-1.2.1/

hadoop fs -put conf input

5.hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+'

6.hadoop fs -cat output/*

7.stop-all.sh

以上步骤中，若都没有出现各种错误，则hadoop安装成功。

 

转载于:https://www.cnblogs.com/cdr-cool/p/3457113.html