获取hadoop度量值到文件的最简单配置

软件

hadoop-2.7.5.tar.gz

jdk-8u171-linux-x64.tar.gz

系统

CentOS Linux release 7.4.1708 (Core)

1.安装 java 和 hadoop

  tar -zxvf jdk-8u171-linux-x64.tar.gz

   mv jdk1.8.0_171/ /opt/jdk

  tar -zxvf hadoop-2.7.5.tar.gz

  mv hadoop-2.7.5/ /opt/hadoop

2.编辑环境变量

vi ~/.bashrc

export JAVA_HOME=/opt/jdk

export HADOOP_HOME=/opt/hadoop

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

3.使环境变量生效

source ~/.bashrc

4.修改 core-site

vi /opt/hadoop/etc/hadoop/core-site.xml

        fs.defaultFS

        hdfs://192.168.14.7:9000

   

5.修改hdfs-site.xml

vi /opt/hadoop/etc/hadoop/hdfs-site.xml

        dfs.replication

        1

   

   

        dfs.namenode.name.dir

        file:///opt/namenode

   

   

        dfs.datanode.data.dir

        file:///opt/datanode

   

6.修改 hadoop-metrics2.properties

vi /opt/hadoop/etc/hadoop/hadoop-metrics2.properties

namenode.sink.file.filename=namenode-metrics.out

datanode.sink.file.filename=datanode-metrics.out

7.新建目录

mkdir /opt/namenode

mkdir /opt/datanode

8.格式化

  hdfs namenode -format

9.启动守护进程

hadoop-daemon.sh start namenode

hadoop-daemon.sh start datanode

10.检查

[root@localhost ~]# ll /opt/hadoop/*out

-rw-r--r-- 1 root root 210996 Apr  1 17:34 /opt/hadoop/datanode-metrics.out

-rw-r--r-- 1 root root 320536 Apr  1 17:34 /opt/hadoop/namenode-metrics.out

[root@localhost ~]# cat /opt/hadoop/namenode-metrics.out |grep -i jvm|tail -n 1

1585733696289 jvm.JvmMetrics: Context=jvm, ProcessName=NameNode, SessionId=null, Hostname=localhost, MemNonHeapUsedM=39.97262, MemNonHeapCommittedM=40.6875, MemNonHeapMaxM=-9.536743E-7, MemHeapUsedM=32.27114, MemHeapCommittedM=58.0625, MemHeapMaxM=966.6875, MemMaxM=966.6875, GcCountCopy=16, GcTimeMillisCopy=93, GcCountMarkSweepCompact=1, GcTimeMillisMarkSweepCompact=39, GcCount=17, GcTimeMillis=132, GcNumWarnThresholdExceeded=0, GcNumInfoThresholdExceeded=0, GcTotalExtraSleepTime=135, ThreadsNew=0, ThreadsRunnable=6, ThreadsBlocked=0, ThreadsWaiting=5, ThreadsTimedWaiting=24, ThreadsTerminated=0, LogFatal=0, LogError=0, LogWarn=2, LogInfo=571

可以看到 jvm 相关的度量值，堆内存，GC等信息