Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and

Azkaban 任务一直处于运行状态,日志显示如下:

12-04-2023 10:12:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 1 of 720
12-04-2023 10:13:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 2 of 720
12-04-2023 10:14:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 3 of 720
12-04-2023 10:15:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 4 of 720
12-04-2023 10:16:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 5 of 720

这是由于 Azkaban 在执行任务时,检测到所有的 executor 节点内存都小于 3GB,其最低要求的内存是 3GB,导致其无法进行任务的分配,一直处于等待的过程。

解决方法:

  • 释放内存;

  • 增加内存;

  • 关闭检测内存的选项;

    • 编辑 $AZKABAN_HOME/azkaban-exec-server/plugins/jobtypes/ 目录下的commonprivate.properties 文件,在其中添加 memCheck.enabled=false 配置项。

你可能感兴趣的:(linux,jvm,Azkaban,大数据,任务调度)