hadoop3.1.1在centos6.5上的伪分布式安装

配置java 环境变量

编辑 /etc/profile 末位增加

export JAVA_HOME=/usr/java/jdk1.8.0_121
export JRE_HOME=/usr/java/jdk1.8.0_121/jre/
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATH

使修改生效

source /etc/profile

配置hdfs

修改hadoop配置文件

修改JAVA_HOME，修改配置文件etc/hadoop/hadoop-env.sh 

export JAVA_HOME=/usr/java/jdk1.8.0_121

修改 etc/hadoop/core-site.xml

    
         fs.defaultFS
         hdfs://localhost:9000
     

修改 etc/hadoop/hdfs-site.xml:

    
          dfs.replication
          1
    
    
          dfs.namenode.name.dir
          /app/appuser/apps/hdata/namenode
    
    
          dfs.datanode.data.dir
          /app/appuser/apps/hdata/datanode
    

配置可以ssh localhost登录机器

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

格式化文件系统

bin/hdfs namenode -format

启动namenode 与datanode

sbin/start-dfs.sh

配置环境变量

再 ~/.bashrc末尾加入

export HADOOP_HOME=/app/appuser/apps/hadoop-3.1.1
export PATH=$HADOOP_HOME/bin:$PATH

hdfs可视化端口 9870

配置yarn

修改 etc/hadoop/mapred-site.xml

    
        mapreduce.framework.name
        yarn
    
    
        mapreduce.application.classpath
        $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*
    

修改 etc/hadoop/yarn-site.xml

    
         yarn.nodemanager.aux-services
         mapreduce_shuffle
    
    
         yarn.nodemanager.env-whitelist
         JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME
    

yarn可视化端口是 8088