hadoop 2.7 配置文件

下面是我测试可用的配置文件，搭建环境见另外一个文章

1. /etc/hadoop/core-site.xml (在系统中建好tmp文件夹)

fs.defaultFS
hdfs://n1:9000


io.file.buffer.size
131072


hadoop.tmp.dir
file:/home/grid/hadoop-2.7.3/tmp
Abase for other temporary directories.


hadoop.proxyuser.hduser.hosts
*


hadoop.proxyuser.hduser.groups
*

2. /etc/hadoop/hdfs-site.xml(按照dir路径，在系统中新建目录)

dfs.namenode.secondary.http-address
n1:9001


dfs.namenode.name.dir
file:/home/grid/hadoop-2.7.3/name


dfs.datanode.data.dir
file:/home/grid/hadoop-2.7.3/data


dfs.replication
2


dfs.webhdfs.enabled
true

3. /etc/hadoop/yarn-site.xml

    
        yarn.nodemanager.aux-services
        mapreduce_shuffle
    
    
        yarn.nodemanager.auxservices.mapreduce.shuffle.class
        org.apache.hadoop.mapred.ShuffleHandler
    
    
        yarn.resourcemanager.address
        n1:8032
    
    
        yarn.resourcemanager.scheduler.address
        n1:8030
    
    
        yarn.resourcemanager.resource-tracker.address
        n1:8031
    
    
        yarn.resourcemanager.admin.address
        n1:8033
    
    
        yarn.resourcemanager.webapp.address
        n1:8088
    
    
        yarn.nodemanager.vmem-check-enabled
        false
    
    
   yarn.nodemanager.vmem-pmem-ratio
    4
    Ratio between virtual memory to physical memory when setting memory limits for containers
 

yarn.nodemanager.resource.memory-mb
1024 
 

yarn.scheduler.minimum-allocation-mb
 128

4. /etc/hadoop/mapred-site.xml(复制模板并修改)

mapreduce.framework.name
yarn


mapreduce.jobhistory.address
n1:10020


mapreduce.jobhistory.webapp.address
n1:19888


    yarn.app.mapreduce.am.resource.mb
    768

5. 修改/hadoop-2.7.3/etc/hadoop目录下hadoop-env.sh、yarn-env.sh的JAVA_HOME（必须修改，用$JAVA_HOME不行）

export JAVA_HOME=/usr/jdk1.8.0_131

6. 修改/etc/hadoop/slaves

d1
d2