mac下安装

一、安装java 8

二、ssh配置

首先确认能够远程登录

系统偏好设置－共享 -远程登录

ssh-keygen -t rsa

Press enter for each line 提示输入直接按回车就好

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

chmod og-wx ~/.ssh/authorized_keys

ssh localhost

如果ssh localhost还需要密码查看一下你.ssh目录的权限

.ssh的父目录的权限问题（我的问题就出现在这里）：.ssh的父目录文件权限应该是755，即所属用户的用户文件（/home下属的一个用户文件）

执行chmod 755 ~/.ssh

三、安装配置hadoop文件

下载地址：http://hadoop.apache.org/releases.html

修改配置文件配置文件在hadoop目录下的/etc/hadoop

1.修改core-site.xml 文件

        fs.defaultFS

        hdfs://localhost:9000



        hadoop.tmp.dir

        /Users/chenxun/software/hadoop-2.8.1/temp

2.修改hdfs-site.xml

默认副本数3，修改为1，dfs.namenode.name.dir指明fsimage存放目录，多个目录用逗号隔开。dfs.datanode.data.dir指定块文件存放目录，多个目录逗号隔开

        dfs.replication

        1





        dfs.namenode.name.dir         file:/Users/chenxun/software/hadoop-2.8.1/tmp/hdfs/name



        dfs.datanode.data.dir         file:/Users/chenxun/software/hadoop-2.8.1/tmp/hdfs/data



        dfs.namenode.secondary.http-address         localhost:9001



        dfs.webhdfs.enabled

        true

3.yarn配置

mapred-site.xml

        mapreduce.framework.name

        yarn





        mapreduce.admin.user.env         HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME



        yarn.app.mapreduce.am.env         HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME



        mapreduce.application.classpath



            /Users/chenxun/software/hadoop-2.8.1/etc/hadoop,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/common/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/common/lib/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/hdfs/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/hdfs/lib/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/mapreduce/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/mapreduce/lib/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/yarn/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/yarn/lib/*

yarn-site.xml

        yarn.nodemanager.aux-services         mapreduce_shuffle





        yarn.application.classpath



            /Users/chenxun/software/hadoop-2.8.1/etc/hadoop,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/common/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/common/lib/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/hdfs/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/hdfs/lib/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/mapreduce/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/mapreduce/lib/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/yarn/*,             /Users/chenxun/software/hadoop-2.8.1/share/hadoop/yarn/lib/*

四、配置hadoop环境变量，可能需要重新编译native library

vim ~/.bash_profile

export HADOOP_HOME=/Users/chenxun/software/hadoop-2.8.1

export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

export LD_LIBRARY_PATH=$HADOOP_HOME/lib/native/

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native:$HADOOP_COMMON_LIB_NATIVE_DIR"

五、运行hadoop 以及hadoop命令简单的介绍

格式化HDFS

hdfs namenode -format

一次启动hadoop所有进程，sbin下：

start-all.sh

打开 http://localhost:50070 进入hdfs管理页面

打开 http://localhost:8088 进入hadoop进程管理页面