flink任务性能优化

1、使用异步算子,异步执行操作

2、将下游数据需要的数据以参数的形式向下传递

3、当服务器资源有限的情况下,慎用RocksDBStateBackend

RocksDBStateBackend performance will be poor because of the current Flink memory configuration! RocksDB will flush memtable constantly, causing high IO and CPU. Typically the easiest fix is to increase task manager managed memory size. If running locally, see the parameter taskmanager.memory.managed.size. Details: arenaBlockSize 8388608 > mutableLimit 3737299 (writeBufferSize = 67108864, arenaBlockSizeConfigured = 0, defaultArenaBlockSize = 8388608, writeBufferManagerCapacity = 4271199)

明确指出,当前内存有限的情况下,使用rocketDB会造成性能损害,因为rocketDB会不停刷内存,造成高io和高cpu。所以在小型化项目时,对状态数据要求不高时,可考虑不使用rocketDB作为状态后端

你可能感兴趣的:(flink,性能优化,大数据)