flink-FlinkFixedPartitioner分区原理

FlinkFixedPartitioner源码:

package org.apache.flink.streaming.connectors.kafka.partitioner;

import org.apache.flink.util.Preconditions;

public class FlinkFixedPartitioner extends FlinkKafkaPartitioner {
    private int parallelInstanceId;

    public FlinkFixedPartitioner() {
    }

    public void open(int parallelInstanceId, int parallelInstances) {
        Preconditions.checkArgument(parallelInstanceId >= 0, "Id of this subtask cannot be negative.");
        Preconditions.checkArgument(parallelInstances > 0, "Number of subtasks must be larger than 0.");
        this.parallelInstanceId = parallelInstanceId;
    }

    public int partition(T record, byte[] key, byte[] value, String targetTopic, int[] partitions) {
        Preconditions.checkArgument(partitions != null && partitions.length > 0, "Partitions of the target topic is empty.");
        return partitions[this.parallelInstanceId % partitions.length];
    }
}

根据flink sink的subtask的id和kafka的partition数量进行取余计算。

flink并行度为3(F0,F1,F2),partition数量为2(P0,P1),则F0->P0,F1->P1,F2->P0

flink并行度为2(F0,F1),partition数量为3(P0,P1,P2),则F0->P0,F1->P1

你可能感兴趣的:(flink)