《日子》.分布式-Hadoop2.6 cluster

一、系统安装及网络环境配制

centos6.5
配制hostname 及 ip
关闭防火墙
3台虚拟机规划
vi /etc/hosts
加入
master.hadoop 192.168.0.10
slave1.hadoop 192.168.0.11
slave2.hadoop 192.168.0.12

二、ssh无密钥登录

参考《日子》.分布式之开篇－SSH无密认证

三、jdk安装及hadoop2.6 拷贝

创建目录 cluster
|- install
|-hadoop-2.6.0

拷贝jdk-7u79-linux-x64.rpm 到install 目录
rpm -ivh jdk-7u79-linux-x64.rpm 安装jdk
拷贝hadoop2.6.0 到cluster目录

四、配制文件修改

core-site.xml







hadoop.tmp.dir
/usr/hadoop/tmp
Abase for other temporary directories.


fs.defaultFS
hdfs://master.hadoop:9000


io.file.buffer.size
4096




vi hadoop-env.sh 和 yarn-env.sh 在开头添加如下环境变量(一定要添加切勿少了)

export JAVA_HOME=/usr/java/jdk1.7.0_79

mapred-site.xml








mapreduce.framework.name
yarn
true


mapreduce.jobtracker.http.address
master.hadoop:50030


mapreduce.jobhistory.address
master.hadoop:10020


mapreduce.jobhistory.webapp.address
master.hadoop:19888


mapred.job.tracker
http://master.hadoop:9001




yarn-site.xml




yarn.resourcemanager.hostname
master.hadoop


yarn.nodemanager.aux-services
mapreduce_shuffle


yarn.resourcemanager.address
master.hadoop:8032


yarn.resourcemanager.scheduler.address
master.hadoop:8030


yarn.resourcemanager.resource-tracker.address
master.hadoop:8031


yarn.resourcemanager.admin.address
master.hadoop:8033


yarn.resourcemanager.webapp.address
master.hadoop:8088




slaves 文件配制

slave1.hadoop
slave2.hadoop

拷贝hadoop2.6.0文件夹到192.168.0.11,192.168.0.12

五、启动集群

格式化namenode
hadoop namenode -formart
启动集群
start-all.sh
访问master

hadoop1.jpg

hadoop2.jpg