hadoop-2.6.0集群安装配置

1.1  hadoop的安装

1.1.1 hadoop需重新编译

我们在官网下载的hadoop是32位的，但是我们的操作系统是64位的（CentOs），所以我们下载源码从新编译，这里有一份编译好的 链接：http://pan.baidu.com/s/1sl97QDv 密码：i2hy

1.1.2 版本介绍

JDK版本:java version "1.7.0_79"

hadoop版本：2.6.0

1.1.3 虚拟机的准备

第一步：准备三台虚拟机

第二步：手动配置服务器的ip地址

配置方式1：可以直接在窗口中手动配置

配置方式2：可以修改/etc/sysconfig/network-scripts/ifcfg-ech2（也可能是 ifcfg-ech0 ）文件

第三步：修改对应ip的虚拟机的名称

执行命令：vim /etc/sysconfig/network

把里面的HOSTNAME等号右边的值修改成:nameNode

第四步：修改vim /etc/hosts文件,添加：ip地址（对应的ip地址） nameNode

例如：192.168.110 nameNode

第五步：重启network服务

执行命令：service network restart

三台虚拟机分别执行上面的步骤得到是的：

192.168.1.110 nameNode

192.168.1.111 dataNode1

192.168.1.112 dataNode2 （如果hostname没有立马生效，执行命令：hostname 主机名 执行命令：hostname nameNode ，还是没有生效，可以重启所有节点）

1.1.4 配置SSH公钥私钥自动登录

在hadoop集群中，nameNode节点，需要能够SSH无密码登录访问dataNode节点。

第一步：进入SSH目录（即为这个目录~）

[root@nameNode ~]# cd ~ [root@nameNode ~]# ll 总用量 84 -rw-------. 1 root root 2180 11月 17 08:14 anaconda-ks.cfg -rw-r--r--. 1 root root 62826 11月 17 08:13 install.log -rw-r--r--. 1 root root 11949 11月 17 08:07 install.log.syslog [root@nameNode ~]#

第二步：生存公钥密钥对

[root@nameNode ~]# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 1e:d0:45:d9:69:c7:d5:8a:53:c7:a3:76:c9:92:c6:2b root@nameNode The key's randomart image is: +--[ RSA 2048]----+ | .oo o oo| | . .. + +.+| | . . ..++oo| | . oB.+ | | S o.+ | | . . E . | | . . | | | | | +-----------------+ [root@nameNode ~]#

第三步：进入到.ssh目录下多出了两个文件

[root@nameNode ~]# cd .ssh [root@nameNode .ssh]# ll 总用量 8 -rw-------. 1 root root 1675 2月 21 21:43 id_rsa -rw-r--r--. 1 root root 395 2月 21 21:43 id_rsa.pub [root@nameNode .ssh]#

私钥文件：id_rsa

公钥文件：id_rsa.pub

第四步：将公钥文件id_rsa.pub内容放到authorized_keys文件中 ：

cat id_rsa.pub >> authorized_keys

第五步：将authorized_keys分发到各个数据节点dataNode中

在这之前需要在nameNode主机上配置数据节点局域网ip地址

即为：

192.168.1.110 nameNode 192.168.1.111 dataNode1 192.168.1.112 dataNode2

 

[root@nameNode .ssh]# scp authorized_keys root@dataNode1:/root/.ssh/ The authenticity of host 'datanode1 (192.168.1.111)' can't be established. RSA key fingerprint is 5a:19:12:ac:62:f9:54:d1:2f:ba:35:94:1f:b2:bc:7b. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'datanode1,192.168.1.111' (RSA) to the list of known hosts. root@datanode1's password: scp: /root/.ssh/: Is a directory [root@nameNode .ssh]#

第六步：验证ssh无密码登录，如下表示成功

[root@nameNode ~]# ssh dataNode1 Last login: Wed Feb 22 22:34:24 2017 from namenode [root@dataNode1 ~]#

1.1.5 hadoop的安装步骤

第一步：使用rz上传jar包hadoop-2.6.0.tar.gz到/export/software/目录下。

第二步：解压hadoop-2.6.0.tar.gz压缩包，到/export/servers/目录下

执行命令：tar -zxvf hadoop-2.6.0.tar.gz -C /export/servers/

第三步：配置环境变量

执行命令：vim /etc/profile

复制下列代码到配置文件中

export HADOOP_HOME=/export/servers/hadoop-2.6.0 export PATH=$PATH:$HADOOP_HOME/bin

让配置文件立马生效

执行命令：source /etc/profile

第四步：进入都/export/servers/hadoop-2.6.0/目录下创建hadoop需要的用到的文件夹，hadoop_save_dir（自己创建的，数据保存目录）。

mkdir -p /export/servers/hadoop-2.6.0/ hadoop_save_dir/tmp

mkdir -p /export/servers/hadoop-2.6.0/ hadoop_save_dir/hdfs

mkdir -p /export/servers/hadoop-2.6.0/ hadoop_save_dir/hdfs/data

mkdir -p /export/servers/hadoop-2.6.0/ hadoop_save_dir/hdfs/name

第五步：进入到/ /export/servers/hadoop-2.6.0/etc/hadoop目录下，配置hadoop-env.sh文件

1. 配置core-site.xml文件

fs.defaultFS hdfs://nameNode:9000 hadoop.tmp.dir /export/servers/hadoop-2.6.0/hadoop_save_dir/hadoop_tmp Abase for other temporary directories. io.file.buffer.size 4096

2. 配置hdfs-site.xml

dfs.nameservices hadoop-cluster1 dfs.namenode.secondary.http-address nameNode:50090 dfs.namenode.name.dir file:///export/servers/hadoop-2.6.0/hadoop_save_dir/dfs/name dfs.datanode.data.dir file:///export/servers/hadoop-2.6.0/hadoop_save_dir/dfs/data dfs.replication 2 dfs.webhdfs.enabled true

3. 配置mapred-site.xml

mapreduce.framework.name yarn mapreduce.jobtracker.http.address nameNode:50030 mapreduce.jobhistory.address nameNode:10020 mapreduce.jobhistory.webapp.address nameNode:19888

4. 配置yarn-site.xml

yarn.nodemanager.aux-services mapreduce_shuffle yarn.resourcemanager.address nameNode:8032 yarn.resourcemanager.scheduler.address nameNode:8030 yarn.resourcemanager.resource-tracker.address nameNode:8031 yarn.resourcemanager.admin.address nameNode:8033 yarn.resourcemanager.webapp.address nameNode:8088

5. 使用vim 打开${hadoop_home}/etc/hadoop/slaves（其实就是和上面的几个配置文件同级目录）文件

向配置文件写入备机名称

dataNode1 dataNode2

6. 修改hadoop-env.sh和yarn-env.sh中添加JAVA_HOME配置

配置hadoop-env.sh:

export JAVA_HOME=/export/servers/jdk

配置yarn-env.sh:

export JAVA_HOME=/export/servers/jdk

1.1.6 hadoop的启动

1. 格式化文件系统 ：（HADOOP_HOME目录下执行，分别执行）

执行命令：bin/hdfs namenode -format

执行命令：bin/hadoop namenode -format(两条命令都需要执行)

输出：

[root@nameNode hadoop-2.6.0]# bin/hdfs namenode –format 17/02/23 18:53:09 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = nameNode/192.168.1.110 STARTUP_MSG: args = [–format] STARTUP_MSG: version = 2.6.0 STARTUP_MSG: classpath = /export/servers/hadoop-2.6.0/etc/hadoop:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/export/servers/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/export/servers/hadoop-2.6.0/contrib/capacity-scheduler/*.jar STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z STARTUP_MSG: java = 1.7.0_79 ************************************************************/ 17/02/23 18:53:09 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 17/02/23 18:53:09 INFO namenode.NameNode: createNameNode [–format] Usage: java NameNode [-backup] | [-checkpoint] | [-format [-clusterid cid ] [-force] [-nonInteractive] ] | [-upgrade [-clusterid cid] [-renameReserved] ] | [-upgradeOnly [-clusterid cid] [-renameReserved] ] | [-rollback] | [-rollingUpgrade ] | [-finalize] | [-importCheckpoint] | [-initializeSharedEdits] | [-bootstrapStandby] | [-recover [ -force] ] | [-metadataVersion ] ] 17/02/23 18:53:09 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at nameNode/192.168.1.110 ************************************************************/ [root@nameNode hadoop-2.6.0]#

2. 启动nameNode和dataNode的守护线程

执行命令：sbin/start-dfs.sh

[root@nameNode hadoop-2.6.0]# sbin/start-dfs.sh 17/02/23 19:23:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Starting namenodes on [nameNode] nameNode: starting namenode, logging to /export/servers/hadoop-2.6.0/logs/hadoop-root-namenode-nameNode.out dataNode2: starting datanode, logging to /export/servers/hadoop-2.6.0/logs/hadoop-root-datanode-dataNode2.out dataNode1: starting datanode, logging to /export/servers/hadoop-2.6.0/logs/hadoop-root-datanode-dataNode1.out 17/02/23 19:23:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable [root@nameNode hadoop-2.6.0]#

3. 启动MapReduceManager和NodeManager守护线程

执行命令：sbin/start-yarn.sh

[root@nameNode hadoop-2.6.0]# sbin/start-yarn.sh starting yarn daemons resourcemanager running as process 7090. Stop it first. dataNode1: starting nodemanager, logging to /export/servers/hadoop-2.6.0/logs/yarn-root-nodemanager-dataNode1.out dataNode2: starting nodemanager, logging to /export/servers/hadoop-2.6.0/logs/yarn-root-nodemanager-dataNode2.out [root@nameNode hadoop-2.6.0]#

4. 查看是否启动成功

执行命令：jps，分别在各个虚拟机中执行jps

7485 DataNode 7794 Jps 7628 NodeManager 7013 DataNode 7155 NodeManager 7309 Jps 8636 ResourceManager 8973 Jps

启动：sbin/start-all.sh

停止：sbin/stop-all.sh