hadoop 3.x 案例1:job history显示问题

一. 问题描述

访问 job history的时候，发现一长串描述，但是没有地方查看对应的日志。

Logs not available for container_1668064306391_0004_01_000001. Aggregation may not be complete, Check back later or try to find the container logs in the local directory of nodemanager hp6:43050

image.png

二. 解决方案

2.1 mapred-site.xml已添加如下配置：

        mapreduce.jobhistory.address
        hadoop000:10020
        MapReduce JobHistory Server IPC host:port



        mapreduce.jobhistory.webapp.address
        hadoop000:19888
        MapReduce JobHistory Server Web UI host:port



        mapreduce.jobhistory.done-dir
        /history/done



        mapreduce.jobhistory.intermediate-done-dir
        /history/done_intermediate

2.2 日志聚合功能已开启：

yarn-site.xml

    yarn.log-aggregation-enable
    true

2.3 job historyserver 已经配置完毕

[hadoop@hadoop000 hadoop]$ jps | grep -v Jps
37602 ResourceManager
4558 SecondaryNameNode
4271 NameNode
4372 DataNode
36310 JobHistoryServer
37699 NodeManager

参考:

1. https://coding.imooc.com/learn/questiondetail/88186.html