Hadoop mac 2.8.4的开发环境搭建

1.java环境安装

网上搜索
JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home

2.下载hadoop2.8.4 binary版本

下载地址：hadoop2.8.4

3.解压文件

$ tar -zxvf hadoop-2.8.0.tar.gz

4.配置hadoop环境变量

4.1 bash_profile文件

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home
export HADOOP_HOME=//Users/yujia/Documents/hadoop/hadoop-2.8.4
export PATH=$PATH:/usr/local/mysql/bin:$HADOOP_HOME/bin:$JAVA_HOME/bin

source ~/.bash_profile

hadoop是否安装成功

$ hadoop version

--显示如下内容：
Hadoop 2.8.4
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r 17e75c2a11685af3e043aa5e604dc831e5b14674
Compiled by jdu on 2018-05-08T02:50Z
Compiled with protoc 2.5.0
From source with checksum b02a59bb17646783210e979bea443b0
This command was run using /Users/yujia/Documents/hadoop/hadoop-2.8.4/share/hadoop/common/hadoop-common-2.8.4.jar

4.2 配置core-site.xml ——(defaultFS)指定了NameNode的主机名与端口

  
<configuration>
     
    <property>  
       <name>fs.defaultFSname> 
       <value>hdfs://localhost:8000value>  
    property>  

     
    <property>  
       <name>hadoop.tmp.dirname> 
       <value>/Users/time/Documents/hadoop-2.7.3/tmpvalue>  
        <description>A base for othertemporary directories.description> 
    property> 
configuration>

4.3 配置hdfs-site.xml——指定了HDFS的默认参数副本数

  
<configuration>  
     
        <property>
          <name>dfs.replicationname>
           <value>1value>
        property>

        
        <property>
            <name>dfs.namenode.name.dirname>
            <value>file:/home/hdfs/namevalue>
        property>

        
        <property>
            <name>dfs.namenode.http-addressname>
             <value>localhost:57000value>
        property>

        <property>
          <name>dfs.datanode.data.dirname>
          <value>file:/home/hdfs/datavalue>
        property>

configuration>

**注意这里的file
如果配置为: file:/path/a/c,则具体的路径就是: /path/a/c
如果配置为:file://path/a/c,则具体的路径就是: /a/c**

4.4 mapred-site.xml配置

$ cp mapred-site.xml.template mapred-site.xml

<property>  
        <name>mapreduce.framework.namename>  
        <value>yarnvalue>  
property>

4.5 Yarn-site.xml 配置

 
 <configuration> 

   
  
  <property> 
  <name>yarn.resourcemanager.addressname>
  <value>localhost:18040value> 
  property> 

  
  <property> 
  <name>yarn.resourcemanager.scheduler.addressname>
  <value>localhost:18030value> 
  property> 

  
  <property> 
  <name>yarn.resourcemanager.resource-tracker.addressname>
  <value> localhost:18025value> 
  property> 

  
  <property> 
  <name>yarn.resourcemanager.webapp.addressname>
  <value>localhost:18088value> 
  property> 

   
  <property> 
  <name>yarn.nodemanager.aux-servicesname>
  <value>mapreduce_shufflevalue> 
  property> 
 configuration>

5.格式化namenode
$ hadoop namenode -format
问题一：碰到找不到java_home的情况
在hadoop安装路径下，进入hadoop/etc/hadoop/目录并编辑hadoop-env.sh，修改JAVA_HOME为JAVA的安装路径
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home

然后执行hadoop namenode -format
问题二：需要输入多次的密码

6.启动hadoop
./sbin/start-all.sh

7.web测试
我们也可以通过浏览器查看hadoop的详细信息，打开链接：

http://localhost:57000/

参考文档：
实战CentOS系统部署Hadoop集群服务
如何在CentOS 7上修改主机名hostname