hadoop2.2.0安装-完整版

之前在centos7安装hadoop2.2.0集群-环境准备及centos7安装hadoop2.2.0集群-软件安装中讲过从最开始的环境准备及软件安装,从设置hostname到jdk的安装都没有问题,只是在这里重新回顾一下完整版的hadoop安装,验证过的,比较靠谱一点。

因为之前jdk安装之前的都没有问题,所以这里只讲hadoop的安装。

1、首先下载hadoop压缩包,hadoop-2.2.0.tar.gz,解压到安装目录下(自己定),然后设置环境变量,如下:


export HADOOP_HOME=/export/servers/hadoop-2.2.0
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"

 

export PATH=$JAVA_HOME/bin:JRE_HOME/bin:$HADOOP_HOME/bin:$PATH

2、修改配置文件,hadoop2.2的配置文件目录是$HADOOP_HOME/etc/hadoop,主要修改的有以下几个配置文件:

core-site.xml:(后两项是设置oozie的代理)



fs.defaultFS
hdfs://hadoop-master


hadoop.tmp.dir
/export/data/hadoop_tmp


hadoop.proxyuser.root.hosts
*


hadoop.proxyuser.root.groups
*

 hadoop-en.sh:

在文件末尾添加java安装目录,例如:export JAVA_HOME=/usr/local/java/jdk1.7.0_71

hdfs-site.xml:



dfs.nameservices
ns1
Logical name for this newnameservice


dfs.ha.namenodes.ns1
nn1
Unique identifiers for each NameNode in the nameservice


dfs.namenode.rpc-address.ns1.nn1
hadoop-master:8020


dfs.namenode.http-address.ns1.nn1
hadoop-master:50070


dfs.replication
1--注释:子节点的个数,超出会报错


dfs.namenode.name.dir
/export/data/nn


dfs.datanode.data.dir
/export/data/dfs

mapred-env.sh:

在文件末尾添加Java安装目录,同上。

mapred-site.xml:



mapreduce.framework.name
yarn


mapreduce.jobhistory.address
hadoop-master:10020


mapreduce.jobhistory.webapp.address
hadoop-master:19888


mapreduce.map.output.compress
false


mapreduce.map.output.compress.codec
org.apache.hadoop.io.compress.GzipCodec


yarn.app.mapreduce.am.staging-dir
/user

slaves文件:

hadoop-slave

yarn-env.sh:

在文件末尾添加Java安装目录,同上。

yarn-site.xml:



yarn.nodemanager.aux-services
mapreduce_shuffle


yarn.nodemanager.aux-services.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler


yarn.resourcemanager.address
hadoop-master:8032


yarn.resourcemanager.scheduler.address
hadoop-master:8030


yarn.resourcemanager.resource-tracker.address
hadoop-master:8031


yarn.resourcemanager.admin.address
hadoop-master:8033


yarn.resourcemanager.webapp.address
hadoop-master:8088


yarn.log-aggregation-enable
true


yarn.nodemanager.log-dirs
/export/data/yarn/logs


yarn.nodemanager.log.retain-seconds
86400


yarn.nodemanager.remote-app-log-dir
/tmp/app-logs


yarn.nodemanager.remote-app-log-dir-suffix
logs


yarn.resourcemanager.scheduler.class
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler


yarn.scheduler.fair.allocation.file
/export/servers/hadoop-2.2.0/etc/hadoop/fair-scheduler.xml


yarn.scheduler.assignmultiple
true


yarn.scheduler.fair.allow-undeclared-pools
false


yarn.scheduler.fair.locality.threshold.node
0.1


yarn.scheduler.fair.locality.threshold.rack
0.1

以上,配置已经完成,现在启动hadoop,第一次启动需要初始化:

hdfs namenode -format

进入$HADOOP_HOME/sbin目录,sh start-all.sh

启动后,在浏览器输入以下地址测试,如图表示启动成功:

http://hadoop-master:8088

hadoop2.2.0安装-完整版_第1张图片

http://hadoop-master:50070

hadoop2.2.0安装-完整版_第2张图片

转载于:https://www.cnblogs.com/zhli/p/4823281.html

你可能感兴趣的:(hadoop2.2.0安装-完整版)