Hadoop-2.7.1单机版安装

阅读更多

一、修改配置(core-site.xml、hdfs-site.xml、mapred-site.xml、yarn-site.xml)

hadoop-2.7.1/etc/hadoop/core-site.xml





   
        fs.default.name
        hdfs://127.0.0.1:9000
   
   
        hadoop.tmp.dir
        file:/tmp
   
   
        io.file.buffer.size
        131702
   


hadoop-2.7.1/etc/hadoop/hdfs-site.xml




   
        dfs.namenode.name.dir
        file:/tmp/dfs/name
   
   
        dfs.datanode.data.dir
        file:/tmp/dfs/data
   
   
        dfs.replication
        1
   
   
        dfs.namenode.secondary.http-address
        127.0.0.1:9001
   
   
        dfs.webhdfs.enabled
        true
   


hadoop-2.7.1/etc/hadoop/mapred-site.xml




   
   
        mapreduce.framework.name
        yarn
   
   
        mapreduce.jobhistory.address
        127.0.0.1:10020
   
   
        mapreduce.jobhistory.webapp.address
        127.0.0.1:19888
   


hadoop-2.7.1/etc/hadoop/yarn-site.xml



   
        yarn.nodemanager.aux-services
        mapreduce_shuffle
   
   
   
        yarn.resourcemanager.webapp.address
        127.0.0.1:8088
   
   


二、配置本机密码登录
输入命令：ssh-keygen后一致按回车健
输入命令：cp   id_rsa.pub    authorized_keys；复制之前检测是否已存在authorized_keys文件，若存在需要将id_rsa.pub的内容复制到 authorized_keys中
另外可能需要服务器上
~/.ssh权限设置为700
~/.ssh/authorized_keys的权限设置为600

三、启动
先格式化：hadoop-2.7.1/bin/hdfs    namenode   -format
启动：hadoop-2.7.1/sbin/start-dfs.sh
启动：hadoop-2.7.1/sbin/start-yarn.sh



四：验证
在浏览器中输入：
http://127.0.0.1:8088


http://127.0.0.1:50070


PS：在Window上启动时出现了如下错误：
org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
时需要下载源代码重新编译，如下:将610行注释掉，直接返回true

若出现下面的错误需要下载winutils.exe文件放入D:\Tools\hadoop-2.7.1\bin目录下
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
15/12/04 19:18:12 ERROR util.Shell: Failed to locate the winutils binary in the
hadoop binary path
java.io.IOException: Could not locate executable D:\Tools\hadoop-2.7.1\bin\winutils.exe in the Hadoop binaries.
        at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356)
        at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371)
        at org.apache.hadoop.util.Shell.(Shell.java:364)
        at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.j
ava:2483)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2508)


hadoop-common-2.2.0-bin-master.zip下载地址：https://codeload.github.com/srccodes/hadoop-common-2.2.0-bin/zip/master

• 
• 大小: 58 KB

• hadoop_dll2.6.0.7z (106.1 KB)
• 下载次数: 0

• hadoop-common-2.2.0-bin-master.zip (272.6 KB)
• 下载次数: 0

• 
• 大小: 130.3 KB

• 
• 大小: 95.8 KB

• 
• 大小: 443.3 KB

• 查看图片附件