CentOS7搭建Hadoop2.6完全分布式集群环境

3台主机

192.168.30.207 Master

192.168.30.251 Node1

192.168.30.252 Node2

直接用root装,省事.

有特殊需求自行add user装.

以下步骤除非特殊说明,否则每台都必须配置.

1.rpm -ivh XX.rpm 装JDK 我这装的 /usr/java/jdk1.8.0_65

   tar -zxvf hadoop-2.6.3.tar.gz 你懂的

   vi /etc/profile 环境变量

export JAVA_HOME=/usr/java/jdk1.8.0_65
export HADOOP_HOME=/hadoop/hadoop-2.6.3
export PATH=$PATH:${HADOOP_HOME}/bin

2.vi /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=master|node1|node2

3. vi /etc/hosts

192.168.30.207 master
192.168.30.251 node1
192.168.30.252 node2

注意:配置文件里面不能有 127.0.0.1 **** 这个配置 否则报错.

4.hostnamectl set-hostname "master"|"node1"|"node2" 修改静态瞬态以及灵活hostname.

5.reboot

6.配置免密码登陆,看下面这个URL:

http://blog.csdn.net/a237428367/article/details/50464153

7.vi /hadoop/hadoop-2.6.3/etc/hadoop/hadoop-env.sh三台都配

export JAVA_HOME=/usr/java/jdk1.8.0_65

8. vi /hadoop/hadoop-2.6.3/etc/hadoop/core-site.xml

        
                hadoop.tmp.dir
                /hadoop/tmp
                Abase for other temporary directories.
        
        
                fs.defaultFS
                hdfs://master:9000
        
        
                io.file.buffer.size
                4096
        

9. vi /hadoop/hadoop-2.6.3/etc/hadoop/hdfs-site.xml

        
                dfs.replication
                2
                nodes total count
        

10. cp /hadoop/hadoop-2.6.3/etc/hadoop/mapred-site.xml.template /hadoop/hadoop-2.6.3/etc/hadoop/mapred-site.xml

     vi /hadoop/hadoop-2.6.3/etc/hadoop/mapred-site.xml

    
        mapreduce.framework.name
        yarn
        true
    
    
        mapreduce.jobtracker.http.address
        master:50030
    
    
        mapreduce.jobhistory.address
        master:10020
    
    
        mapreduce.jobhistory.webapp.address
        master:19888
    
    
         mapred.job.tracker
         http://master:9001
    

11. vi /hadoop/hadoop-2.6.3/etc/hadoop/yarn-site.xml

    
         yarn.resourcemanager.hostname
         master
    
    
        yarn.nodemanager.aux-services
        mapreduce_shuffle
    
    
        yarn.resourcemanager.address
        master:8032
    
    
        yarn.resourcemanager.scheduler.address
        master:8030
    
    
        yarn.resourcemanager.resource-tracker.address
        master:8031
    
    
        yarn.resourcemanager.admin.address
        master:8033
    
    
        yarn.resourcemanager.webapp.address
        master:8088
    

12. vi /hadoop/hadoop-2.6.3/etc/hadoop/slaves

node1
node2

13.格式化(只在master)

cd /hadoop/hadoop-2.6.3/bin/

./hadoop namenode -format

14.启动(只在master)

cd /hadoop/hadoop-2.6.3/sbin

./start-all.sh

几个yes yes yes yes

15.查看各主机进程

jps 之后如下:

master:

4080 Jps
3649 SecondaryNameNode
3462 NameNode
3798 ResourceManager

node1:

3173 DataNode
3269 NodeManager
3389 Jps

node2:

3062 DataNode
3270 Jps
3149 NodeManager

然后恭喜,安装完毕.

如果50070和8088页面打不开请检查防火墙

systemctl stop firewalld.service

systemctl disable firewalld.service

自选

最后,如果想停止,则:

cd /hadoop/hadoop-2.6.3/sbin

./stop-all.sh