skywalking源码--轻量级队列内核
本源码来自于skywalking-agent 8.9.0版本
本节主要讲解skywalking-agent的轻量级队列内核，该实现在datacarrier模块主要用于进行数据发送OAP服务端的实现，主要采用缓存批量异步发送的方式进行发送。
注：本篇文章主要是作为自己看书后的总结，内容有可能会存在一些个人理解上的偏差，如果有网友找出问题欢迎提出，感谢！！！如果我理解上的错误误导了您，在此表示抱歉！！！
文章目录
-
- 相关包及类
-
- 大体的架构图如下
- 源码分析
-
- buffer 包
-
- QueueBuffer接口
- Buffer
-
- 相关属性|方法解释
- Channels
-
- 相关属性|方法解释
- partition包
-
- IDataPartitioner
- SimpleRollingPartitioner
- ProducerThreadPartitioner
- consumer包
-
- IConsumer接口
- ConsumerThread
- ConsumeDriver
- DataCarrier
相关包及类
大体的架构图如下
buffer为一个环形数组，存在一个index属性用于记录当前数据存储的位置，由于数据存储存在存储失败的可能，所以需要根据存储策略做具体的存储；channels为buffer的封装，一个channels内部存在多个buffer数组，所以生产者将数据发送到 channels，channels根据指定的分区实现类进行数据存储到具体分区的计算，由于存储存在失败所以依然需要根据存储策略做相应处理。
源码分析
buffer 包
QueueBuffer接口
public interface QueueBuffer<T> {
/**
* Save data into the queue;
*
* @param data to add.
* @return true if saved
*/
boolean save(T data);
/**
* Set different strategy when queue is full.
*/
void setStrategy(BufferStrategy strategy);
/**
* Obtain the existing data from the queue
*/
void obtain(List<T> consumeList);
int getBufferSize();
}
QueueBuffer为buffer实现的接口，这里的save方法是核心，里面涉及到保存策略
Buffer
public class Buffer<T> implements QueueBuffer<T> {
private final Object[] buffer;
private BufferStrategy strategy;
private AtomicRangeInteger index;
Buffer(int bufferSize, BufferStrategy strategy) {
buffer = new Object[bufferSize];
this.strategy = strategy;
index = new AtomicRangeInteger(0, bufferSize);
}
@Override
public void setStrategy(BufferStrategy strategy) {
this.strategy = strategy;
}
@Override
public boolean save(T data) {
int i = index.getAndIncrement();
// **buffer 写入位置，暂未被消费，已经存在值,这时候根据使用的存储策略进行处理**
if (buffer[i] != null) {
switch (strategy) {
case IF_POSSIBLE:
return false;
default:
}
}
buffer[i] = data;
return true;
}
@Override
public int getBufferSize() {
return buffer.length;
}
@Override
public void obtain(List<T> consumeList) {
this.obtain(consumeList, 0, buffer.length);
}
void obtain(List<T> consumeList, int start, int end) {
for (int i = start; i < end; i++) {
if (buffer[i] != null) {
consumeList.add((T) buffer[i]);
buffer[i] = null;
}
}
}
}
相关属性|方法解释
buffer 属性，缓冲数组。Producer 保存的数据到 buffer 里。
strategy ，缓冲策略 。
index 属性，递增位置。
save()方法：Buffer 在保存数据时，把 buffer 作为一个 “环”，使用 index 记录最后存储的位置，不断向下，循环存储到 buffer 中。通过这样的方式，带来良好的存储性能，避免扩容问题。但是这样的存储会存在冲突的问题：buffer 写入位置，暂未被消费，已经存在值。此时，根据不同的 BufferStrategy 进行处理。整体流程见在save(data) 方法内部。
obtain()方法：当 Buffer 被 Consumer 消费时，被调用 #obtain(start, end) 方法，获得数据并清空。为什么会带 start 、end 方法参数呢？下文揭晓答案。
Channels
public class Channels<T> {
private final QueueBuffer<T>[] bufferChannels;
private IDataPartitioner<T> dataPartitioner;
private final BufferStrategy strategy;
private final long size;
public Channels(int channelSize, int bufferSize, IDataPartitioner<T> partitioner, BufferStrategy strategy) {
this.dataPartitioner = partitioner;
this.strategy = strategy;
bufferChannels = new QueueBuffer[channelSize];
for (int i = 0; i < channelSize; i++) {
if (BufferStrategy.BLOCKING.equals(strategy)) {
bufferChannels[i] = new ArrayBlockingQueueBuffer<>(bufferSize, strategy);
} else {
bufferChannels[i] = new Buffer<>(bufferSize, strategy);
}
}
// noinspection PointlessArithmeticExpression
size = 1L * channelSize * bufferSize; // it's not pointless, it prevents numeric overflow before assigning an integer to a long
}
public boolean save(T data) {
int index = dataPartitioner.partition(bufferChannels.length, data);
int retryCountDown = 1;
// **当缓冲策略为 BufferStrategy.IF_POSSIBLE 时，根据 IDataPartitioner 定义的重试次数，进行多次保存数据直到成功**
if (BufferStrategy.IF_POSSIBLE.equals(strategy)) {
int maxRetryCount = dataPartitioner.maxRetryCount();
if (maxRetryCount > 1) {
retryCountDown = maxRetryCount;
}
}
for (; retryCountDown > 0; retryCountDown--) {
if (bufferChannels[index].save(data)) {
return true;
}
}
return false;
}
public void setPartitioner(IDataPartitioner<T> dataPartitioner) {
this.dataPartitioner = dataPartitioner;
}
/**
* override the strategy at runtime. Notice, this will override several channels one by one. So, when running
* setStrategy, each channel may use different BufferStrategy
*/
public void setStrategy(BufferStrategy strategy) {
for (QueueBuffer<T> buffer : bufferChannels) {
buffer.setStrategy(strategy);
}
}
/**
* get channelSize
*/
public int getChannelSize() {
return this.bufferChannels.length;
}
public long size() {
return size;
}
public QueueBuffer<T> getBuffer(int index) {
return this.bufferChannels[index];
}
}
相关属性|方法解释
bufferChannels 属性，Buffer 数组。
dataPartitioner 属性，数据分区。
strategy 属性，缓冲策略。
save()方法：Channels 在保存数据时，相比 Buffer ，从 buffer 变成了多 buffer ，因此需要先选一个 buffer 。通过使用不同的 IDataPartitioner 实现类，进行 Buffer 的选择。当缓冲策略为 BufferStrategy.IF_POSSIBLE 时，根据 IDataPartitioner 定义的重试次数，进行多次保存数据直到成功。整体流程见 save(data) 方法。
partition包
该包下存在 IDataPartitioner 接口以及它的两个实现类ProducerThreadPartitioner、SimpleRollingPartitioner
IDataPartitioner
public interface IDataPartitioner<T> {
int partition(int total, T data);
/**
* @return an integer represents how many times should retry when {@link BufferStrategy#IF_POSSIBLE}.
*
* Less or equal 1, means not support retry.
*/
int maxRetryCount();
}
partition(total, data) 接口方法，获得数据被分配的分区位置。
maxRetryCount() 接口方法，获得最大重试次数。
SimpleRollingPartitioner
public class SimpleRollingPartitioner<T> implements IDataPartitioner<T> {
// **用于记录下一个当前分配的分区号**
@SuppressWarnings("NonAtomicVolatileUpdate")
private volatile int i = 0;
@Override
public int partition(int total, T data) {
return Math.abs(i++ % total);
}
@Override
public int maxRetryCount() {
return 3;
}
}
SimpleRollingPartitioner ，基于顺序分配策略的数据分配者实现类。
ProducerThreadPartitioner
public class ProducerThreadPartitioner<T> implements IDataPartitioner<T> {
public ProducerThreadPartitioner() {
}
@Override
public int partition(int total, T data) {
return (int) Thread.currentThread().getId() % total;
}
@Override
public int maxRetryCount() {
return 1;
}
}
ProducerThreadPartitioner ，基于线程编号分配策略的数据分配者实现类。
consumer包
主要包含 ConsumeDriver 、ConsumerThread 、IConsumer 三个类。
ConsumerThread 使用 IConsumer ，消费数据
ConsumeDriver 是 ConsumerThread 的线程池封装
IConsumer接口
消费者接口。定义了如下方法：
init() 接口方法，初始化消费者。
consume(List) 接口方法，批量消费消息。
onError(List, Throwable) 接口方法，处理当消费发生异常。
onExit() 接口方法，处理当消费结束。此处的结束时，ConsumerThread 关闭。
ConsumerThread
继承 java.lang.Thread ，消费线程。
public class ConsumerThread<T> extends Thread {
private volatile boolean running;
private IConsumer<T> consumer;
private List<DataSource> dataSources;
private long consumeCycle;
ConsumerThread(String threadName, IConsumer<T> consumer, long consumeCycle) {
super(threadName);
this.consumer = consumer;
running = false;
dataSources = new ArrayList<DataSource>(1);
this.consumeCycle = consumeCycle;
}
/**
* add whole buffer to consume
*/
void addDataSource(QueueBuffer<T> sourceBuffer) {
this.dataSources.add(new DataSource(sourceBuffer));
}
@Override
public void run() {
running = true;
final List<T> consumeList = new ArrayList<T>(1500);
// **不断消费，直到线程关闭( #shutdown() )**
while (running) {
// consume(consumeList):批量消费数据
if (!consume(consumeList)) {
try {
// **当未消费到数据，说明 dataSources 为空，等待 consumeCycle ms ，避免 CPU 空跑。**
Thread.sleep(consumeCycle);
} catch (InterruptedException e) {
}
}
}
// consumer thread is going to stop
// consume the last time
// **当线程关闭，调用 consume() 方法，消费完 dataSources 剩余的数据**
consume(consumeList);
// onExit() 方法，处理当消费结束
consumer.onExit();
}
private boolean consume(List<T> consumeList) {
for (DataSource dataSource : dataSources) {
dataSource.obtain(consumeList);
}
// 从 dataSources 中，获取要消费的数据
if (!consumeList.isEmpty()) {
try {
// 当有数据可消费时，调用 consume(List) 方法。
consumer.consume(consumeList);
} catch (Throwable t) {
// 当消费发生异常时，调用 onError(List, Throwable) 方法
consumer.onError(consumeList, t);
} finally {
consumeList.clear();
}
return true;
}
consumer.nothingToConsume();
return false;
}
void shutdown() {
running = false;
}
/**
* DataSource is a refer to {@link Buffer}.
*/
class DataSource {
private QueueBuffer<T> sourceBuffer;
DataSource(QueueBuffer<T> sourceBuffer) {
this.sourceBuffer = sourceBuffer;
}
void obtain(List<T> consumeList) {
sourceBuffer.obtain(consumeList);
}
}
}
running 属性，是否运行中。
consumer 属性，消费者对象。
dataSources 属性，消费消息的数据源( DataSource )数组。一个 ConsumerThread ，可以消费多个 Buffer ，并且单个 Buffer 消费的分区范围可配置，即一个 Buffer 可以被多个 ConsumerThread 同时无冲突的消费。
addDataSource(sourceBuffer, start, end) 方法，添加 Buffer 部分范围。
addDataSource(sourceBuffer) 方法，添加 Buffer 全部范围。
ConsumeDriver
消费者驱动，提供了对 Channels 启动指定数量的 ConsumerThread 进行消费。
public class ConsumeDriver<T> implements IDriver {
private boolean running;
private ConsumerThread[] consumerThreads;
private Channels<T> channels;
private ReentrantLock lock;
public ConsumeDriver(String name,
Channels<T> channels, Class<? extends IConsumer<T>> consumerClass,
int num,
long consumeCycle,
Properties properties) {
this(channels, num);
for (int i = 0; i < num; i++) {
consumerThreads[i] = new ConsumerThread(
"DataCarrier." + name + ".Consumer." + i + ".Thread", getNewConsumerInstance(consumerClass, properties),
consumeCycle
);
consumerThreads[i].setDaemon(true);
}
}
public ConsumeDriver(String name, Channels<T> channels, IConsumer<T> prototype, int num, long consumeCycle) {
this(channels, num);
prototype.init(new Properties());
for (int i = 0; i < num; i++) {
consumerThreads[i] = new ConsumerThread(
"DataCarrier." + name + ".Consumer." + i + ".Thread", prototype, consumeCycle);
consumerThreads[i].setDaemon(true);
}
}
private ConsumeDriver(Channels<T> channels, int num) {
running = false;
this.channels = channels;
consumerThreads = new ConsumerThread[num];
lock = new ReentrantLock();
}
private IConsumer<T> getNewConsumerInstance(Class<? extends IConsumer<T>> consumerClass, Properties properties) {
try {
IConsumer<T> inst = consumerClass.getDeclaredConstructor().newInstance();
inst.init(properties);
return inst;
} catch (InstantiationException e) {
throw new ConsumerCannotBeCreatedException(e);
} catch (IllegalAccessException e) {
throw new ConsumerCannotBeCreatedException(e);
} catch (NoSuchMethodException e) {
throw new ConsumerCannotBeCreatedException(e);
} catch (InvocationTargetException e) {
throw new ConsumerCannotBeCreatedException(e);
}
}
@Override
public void begin(Channels channels) {
// **正在运行中，直接返回**
if (running) {
return;
}
// 获得锁
lock.lock();
try {
// **调用 #allocateBuffer2Thread() 方法，将 channels 的多个 Buffer ，分配给 consumerThreads 的多个 ConsumerThread。**
this.allocateBuffer2Thread();
for (ConsumerThread consumerThread : consumerThreads) {
// **启动每个 ConsumerThread ，开始消费。**
consumerThread.start();
}
// **标记正在运行中**
running = true;
} finally {
// **释放锁**
lock.unlock();
}
}
@Override
public boolean isRunning(Channels channels) {
return running;
}
private void allocateBuffer2Thread() {
int channelSize = this.channels.getChannelSize();
/**
* if consumerThreads.length < channelSize
* each consumer will process several channels.
*
* if consumerThreads.length == channelSize
* each consumer will process one channel.
*
* if consumerThreads.length > channelSize
* there will be some threads do nothing.
*/
// **轮巡将每个channel分配给consumerThread，这里注意如果consumerThread大于channel那么会出现大于channel的那部分channel会没有channel可以消费。如果channel大于consumerThread，那么编号位于前面的consumerThread将多消费channel。**
for (int channelIndex = 0; channelIndex < channelSize; channelIndex++) {
int consumerIndex = channelIndex % consumerThreads.length;
consumerThreads[consumerIndex].addDataSource(channels.getBuffer(channelIndex));
}
}
@Override
public void close(Channels channels) {
// **获得锁**
lock.lock();
try {
// **标记不在运行中**
this.running = false;
for (ConsumerThread consumerThread : consumerThreads) {
// **关闭每个 ConsumerThread ，结束消费**
consumerThread.shutdown();
}
} finally {
// **释放锁**
lock.unlock();
}
}
}
running 属性，是否运行中。
consumerThreads 属性，ConsumerThread 数组，通过构造方法的 num 参数进行指定。
channels 属性，数据通道。
lock 属性，锁。保证 ConsumerPool 启动或关闭时的线程安全。
begin() 方法，启动 ConsumerPool ，进行数据消费。
close() 方法，关闭 ConsumerPool 。
DataCarrier
DataCarrier 异步处理库的入口程序。通过创建 DataCarrier 对象，使用生产者消费者的模式，执行异步执行逻辑。它就是将上面的 consumer、channel、partition整合在一起的数据搬运工类
public class ConsumeDriver<T> implements IDriver {
private boolean running;
private ConsumerThread[] consumerThreads;
private Channels<T> channels;
private ReentrantLock lock;
public ConsumeDriver(String name,
Channels<T> channels, Class<? extends IConsumer<T>> consumerClass,
int num,
long consumeCycle,
Properties properties) {
this(channels, num);
for (int i = 0; i < num; i++) {
consumerThreads[i] = new ConsumerThread(
"DataCarrier." + name + ".Consumer." + i + ".Thread", getNewConsumerInstance(consumerClass, properties),
consumeCycle
);
consumerThreads[i].setDaemon(true);
}
}
public ConsumeDriver(String name, Channels<T> channels, IConsumer<T> prototype, int num, long consumeCycle) {
this(channels, num);
prototype.init(new Properties());
for (int i = 0; i < num; i++) {
consumerThreads[i] = new ConsumerThread(
"DataCarrier." + name + ".Consumer." + i + ".Thread", prototype, consumeCycle);
consumerThreads[i].setDaemon(true);
}
}
private ConsumeDriver(Channels<T> channels, int num) {
running = false;
this.channels = channels;
consumerThreads = new ConsumerThread[num];
lock = new ReentrantLock();
}
private IConsumer<T> getNewConsumerInstance(Class<? extends IConsumer<T>> consumerClass, Properties properties) {
try {
IConsumer<T> inst = consumerClass.getDeclaredConstructor().newInstance();
inst.init(properties);
return inst;
} catch (InstantiationException e) {
throw new ConsumerCannotBeCreatedException(e);
} catch (IllegalAccessException e) {
throw new ConsumerCannotBeCreatedException(e);
} catch (NoSuchMethodException e) {
throw new ConsumerCannotBeCreatedException(e);
} catch (InvocationTargetException e) {
throw new ConsumerCannotBeCreatedException(e);
}
}
@Override
public void begin(Channels channels) {
if (running) {
return;
}
lock.lock();
try {
this.allocateBuffer2Thread();
for (ConsumerThread consumerThread : consumerThreads) {
consumerThread.start();
}
running = true;
} finally {
lock.unlock();
}
}
@Override
public boolean isRunning(Channels channels) {
return running;
}
private void allocateBuffer2Thread() {
int channelSize = this.channels.getChannelSize();
/**
* if consumerThreads.length < channelSize
* each consumer will process several channels.
*
* if consumerThreads.length == channelSize
* each consumer will process one channel.
*
* if consumerThreads.length > channelSize
* there will be some threads do nothing.
*/
for (int channelIndex = 0; channelIndex < channelSize; channelIndex++) {
int consumerIndex = channelIndex % consumerThreads.length;
consumerThreads[consumerIndex].addDataSource(channels.getBuffer(channelIndex));
}
}
@Override
public void close(Channels channels) {
lock.lock();
try {
this.running = false;
for (ConsumerThread consumerThread : consumerThreads) {
consumerThread.shutdown();
}
} finally {
lock.unlock();
}
}
}