hadoop配置

腾讯云中伪分布式配置:
首先给主机定义一个名称:注意这里需要配置本机的内网机器,其它机器的外网地址

10.104.222.163 hadoopmaster
127.0.0.1 VM_222_163_centos VM_222_163_centos
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 VM_222_163_centos VM_222_163_centos
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

hadoop安装目录假定为${HADOOOP_HOME},当前hadoop版本为2.9.1:

hadoop版本

1 在${HADOOOP_HOME}/etc/hadoop目录下,修改下面几个文件:
core-site.xml




    fs.defaultFS
    hdfs://hadoopmaster:9000



    hadoop.tmp.dir
    /usr/local/hadoop/hadoop-2.9.1/hadoop


hdfs-site.xml



    dfs.name.dir
    /usr/local/hadoop/hdfs/name
    namenode上存储hdfs名字空间元数据 



    dfs.data.dir
    /usr/local/hadoop/hdfs/data
    datanode上数据块的物理存储位置




    dfs.replication
    1


通过拷贝生成mapred-site.xml

 cp mapred-site.xml.template mapred-site.xml 

内容如下:



        
                mapreduce.framework.name
                yarn
        

yarn-site.xml



     
                 yarn.acl.enable
                 0
    
    
        yarn.nodemanager.aux-services
        mapreduce_shuffle
    
    
        yarn.nodemanager.aux-services.mapreduce.shuffle.class
        org.apache.hadoop.mapred.ShuffleHandler
    
    
        yarn.resourcemanager.hostname
        hadoopmaster
    

启动hdfs

${HADOOOP_HOME}/sbin/start-dfs.sh

启动yarn

${HADOOOP_HOME}/sbin/start-yarn.sh

检查hadoop相关进程启动情况:


hadoop进程

如果想要关闭hadoop进程,可以执行:

${HADOOOP_HOME}/sbin/stop-dfs.sh
${HADOOOP_HOME}/sbin/stop-yarn.sh

web中查看hadoop状态:http://outerIP:50070

hadoop状态

web中查看集群中应用程序状态:http://outerIP:8088
集群状态

你可能感兴趣的:(hadoop配置)