spark中日志清理

spark开启history-server之后,app日志会保存在制定的目录下,

若yarn开启日志收集,也需要进行app-logs的自动清理

## yarn日志收集
yarn.log-aggregation.retain-seconds = 1209600
yarn.log-aggregation.retain-check-interval-seconds = 86400


## spark-history日志
spark.history.fs.cleaner.enabled = true
spark.history.fs.cleaner.interval = 1d
spark.history.fs.cleaner.maxAge = 14d

## spark2-history日志
spark.history.fs.cleaner.enabled = true
spark.history.fs.cleaner.interval = 1d
spark.history.fs.cleaner.maxAge = 14d

 

你可能感兴趣的:(spark)