一、前言
本文主要内容
1、bufferqueue 的四大流程；
2、简要介绍四大流程的调用过程；
本文主要讲解安卓图形绘制中的buffer queue
页面数据通过canvas/Open GL ES/vulkan绘制后，形成一帧buffer，交给buffer queue，buffer queue再把数据送到屏幕上显示。所以buffer queue是图形绘制中很核心的一环，它本身就是一个缓冲区池与队列相结合的数据结构。
Buffer queue:图像缓冲区，系统中绝大部分图形数据都会缓冲于此。等待surfaceflinger组织进一步的合成和显示
备注：本文摘录的代码只摘录关键代码和路径，需读者自行结合阅读.由于本文的重点在bufferqueue，其它和bufferqueue相关的流程提到的时候会精简列出，作为辅助理解。
二、Buffer queue
2.1、buffer queue的创建
bufferquque构建
//BufferQueue.cpp frameworks\native\libs\gui
void BufferQueue::createBufferQueue(sp* outProducer,
sp* outConsumer,
bool consumerIsSurfaceFlinger) {
sp core(new BufferQueueCore());
sp producer(new BufferQueueProducer(core, consumerIsSurfaceFlinger));
sp consumer(new BufferQueueConsumer(core));
*outProducer = producer;
*outConsumer = consumer;
}
构建调用流程
下面堆栈表示了底层surface创建bufferqueue的一个过程
04-20 11:45:54.739 674 707 E createBufferQueue: #00 pc 0004ca33 /system/lib/libgui.so (android::BufferQueue::createBufferQueue(android::sp*, android::sp*, bool)+74)
04-20 11:45:54.739 674 707 E createBufferQueue: #01 pc 00068539 /system/lib/libsurfaceflinger.so (android::BufferQueueLayer::onFirstRef()+40)
04-20 11:45:54.739 674 707 E createBufferQueue: #02 pc 000d5ad1 /system/lib/libsurfaceflinger.so (android::surfaceflinger::DefaultFactory::createBufferQueueLayer(android::LayerCreationArgs const&)+52)
04-20 11:45:54.739 674 707 E createBufferQueue: #03 pc 000c11d5 /system/lib/libsurfaceflinger.so (android::SurfaceFlinger::createLayer(android::String8 const&, android::sp const&, unsigned int, unsigned int, int, unsigned int, android::LayerMetadata, android::sp*, android::sp*, android::sp const&, android::sp const&, unsigned int*)+1056)
04-20 11:45:54.739 674 707 E createBufferQueue: #04 pc 0006e197 /system/lib/libsurfaceflinger.so (android::Client::createSurface(android::String8 const&, unsigned int, unsigned int, int, unsigned int, android::sp const&, android::LayerMetadata, android::sp*, android::sp*, unsigned int*)+122)
04-20 11:45:54.739 674 707 E createBufferQueue: #05 pc 0007b34f /system/lib/libgui.so (int android::SafeBnInterface::MethodCaller const&, android::LayerMetadata, android::sp*, android::sp*, unsigned int*> >::callHelper, int (android::ISurfaceComposerClient::*)(android::String8 const&, unsigned int, unsigned int, int, unsigned int, android::sp const&, android::LayerMetadata, android::sp*, android::sp*, unsigned int*), std::__1::tuple, android::LayerMetadata, android::sp, android::sp, unsigned int>, 0u, 1u, 2u, 3u, 4u, 5u, 6u, 7u, 8u, 9u>(android
04-20 11:45:54.739 674 707 E createBufferQueue: #06 pc 00079e2f /system/lib/libgui.so (int android::SafeBnInterface::callLocal const&, android::LayerMetadata, android::sp*, android::sp*, unsigned int*)>(android::Parcel const&, android::Parcel*, int (android::ISurfaceComposerClient::*)(android::String8 const&, unsigned int, unsigned int, int, unsigned int, android::sp const&, android::LayerMetadata, android::sp*, android::sp*, unsigned int*))+158)
04-20 11:45:54.739 674 707 E createBufferQueue: #07 pc 00079d3f /system/lib/libgui.so (android::BnSurfaceComposerClient::onTransact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+46)
BufferQueueCore:bufferqueue的实际实例，由他来衔接生产者和消费者
BufferQueueProducer:bufferqueue工厂模式的接口，生产者
BufferQueueConsumer:bufferqueue消费接口，消费者
BufferQueueCore 负责维护 BufferQueue 的基本数据结构，而 BufferQueueProducer 和 BufferQueueConsumer 则负责提供操作 BufferQueue 的基本接口。
2.2、buffer queue整体流程
2.2.1、流程概要
image.png
这张图片清晰的表述的buffer queue的工作方式，两大功能：生产buffer、消费buffer。
dequeueBuffer:向buffer queue请求一个缓冲区，并指定缓冲区的宽度、高度、像素格式和用法标志
queueBuffer:把缓冲区返回到队列
acquireBuffer:获取缓冲区内容
releaseBuffer:把数据返回到队列
这里有两次返回队列，其实都是返回buffer queue的数据结构中。简述两个流程
1、dequeue、queue等于先申请缓冲区，并告诉producer申请成功，producer就把数据queue给buffer queue数据结构中
2、当consumer要的时候先去acquere获取缓冲区内容，获取后进行合成，合成完了需要释放，就给回buffer queue来释放
3、buffer queue操作数据时不会复制一份数据，只会操作数据本身。
通篇文章，我们就在理解BufferQueueProducer 和 BufferQueueConsumer如何操作上面这四个流程
requestBuffer
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {
status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence, reqWidth, reqHeight,
reqFormat, reqUsage, &mBufferAge,
enableFrameTimestamps ? &frameTimestamps
: nullptr);
...
if ((result & IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION) || gbuf == nullptr) {
if (mReportRemovedBuffers && (gbuf != nullptr)) {
mRemovedBuffers.push_back(gbuf);
}
result = mGraphicBufferProducer->requestBuffer(buf, &gbuf);
requestBuffer单独提一下这个流程，当我们首次dequeueBuffer过后，surface会调用requestBuffer,requestBuffer的主要作用就是把GraphicBuffer传给应用侧，
这个动作只需要做一次，应用侧拿到GraphicBuffer过后就可以把共享内存mmap到应用侧的内存空间，然后才可以绘制图像。
2.2.2、bufferqueue被调用流程
本小结主要目的是，在我们认识bufferqueu的是如何工作的同时，不脱离surfaceflinger整体框架。简单提一下它是如果调用来的。以便我们更深入理解。
1、dequeueBuffer&&queueBuffer
当app创建window的时候，会创建surface来显示。实则会创建底层的surface。底层surface创建时会hook住dequeueBuffer和queueBuffer方法。
surface持有mGraphicBufferProducer对象来创建bufferqueue
dequeueBuffer
方法位置：BufferQueueProducer::dequeueBuffer
04-20 09:48:38.909 6053 6263 E graphic dequeueBuffer: #00 pc 0006dea5 /system/lib/libgui.so (android::BpGraphicBufferProducer::dequeueBuffer(int*, android::sp*, unsigned int, unsigned int, int, unsigned long long, unsigned long long*, android::FrameEventHistoryDelta*)+136)
04-20 09:48:38.909 6053 6263 E graphic dequeueBuffer: #01 pc 000842ed /system/lib/libgui.so (android::Surface::dequeueBuffer(ANativeWindowBuffer**, int*)+316)
04-20 09:48:38.909 6053 6263 E graphic dequeueBuffer: #02 pc 0015004b /system/lib/libhwui.so (android::uirenderer::renderthread::ReliableSurface::hook_dequeueBuffer(ANativeWindow*, int (*)(ANativeWindow*, ANativeWindowBuffer**, int*), void*, ANativeWindowBuffer**, int*)+42)
04-20 09:48:38.909 6053 6263 E graphic dequeueBuffer: #03 pc 00083181 /system/lib/libgui.so (android::Surface::hook_dequeueBuffer(ANativeWindow*, ANativeWindowBuffer**, int*)+60)
//接上面，binder调用到BufferQueueProducer
04-20 09:48:38.946 668 697 E dequeueBuffer: #00 pc 00052653 /system/lib/libgui.so (android::BufferQueueProducer::dequeueBuffer(int*, android::sp*, unsigned int, unsigned int, int, unsigned long long, unsigned long long*, android::FrameEventHistoryDelta*)+250)
04-20 09:48:38.946 668 697 E dequeueBuffer: #01 pc 0006d117 /system/lib/libgui.so (android::BnGraphicBufferProducer::onTransact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+374)
queueBuffer
方法位置：BufferQueueProducer::queueBuffer
04-20 09:48:38.987 6053 6263 E queueBuffer: #00 pc 0006e4e1 /system/lib/libgui.so (android::BpGraphicBufferProducer::queueBuffer(int, android::IGraphicBufferProducer::QueueBufferInput const&, android::IGraphicBufferProducer::QueueBufferOutput*)+128)
04-20 09:48:38.987 6053 6263 E queueBuffer: #01 pc 00084d83 /system/lib/libgui.so (android::Surface::queueBuffer(ANativeWindowBuffer*, int)+722)
04-20 09:48:38.987 6053 6263 E queueBuffer: #02 pc 00083259 /system/lib/libgui.so (android::Surface::hook_queueBuffer(ANativeWindow*, ANativeWindowBuffer*, int)+60)
//接上面，binder调用到BufferQueueProducer
04-20 09:48:39.017 668 697 E queueBuffer: #00 pc 00053877 /system/lib/libgui.so (android::BufferQueueProducer::queueBuffer(int, android::IGraphicBufferProducer::QueueBufferInput const&, android::IGraphicBufferProducer::QueueBufferOutput*)+266)
04-20 09:48:39.017 668 697 E queueBuffer: #01 pc 0006d345 /system/lib/libgui.so (android::BnGraphicBufferProducer::onTransact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)+932)
2、acquireBuffer&&releaseBuffer
1、我们知道系统的刷新由Vsync信号控制，一个Vsync信号来时。surfaceflinger代码会走INVALIDATE/REFRESH,最终都是走refresh
2、下面接收vsync的MessageQueue队列，实际就是surfaceflinger初始化时创建的mEventQueue。
3、EventThread通过Connection通知VSYNC信号到达时可以触发回调通知MessageQueue
4、vsync这个流程对应bufferqueue流程中的acquireBuffer、releaseBuffer，调用流程堆栈下面已列出。
//SurfaceFlinger.cpp
void SurfaceFlinger::onMessageReceived(int32_t what, nsecs_t expectedVSyncTime) {
ATRACE_CALL();
switch (what) {
case MessageQueue::INVALIDATE: {
onMessageInvalidate(expectedVSyncTime);
break;
}
case MessageQueue::REFRESH: {
onMessageRefresh();
break;
}
}
}
acquireBuffer
方法位置：BufferQueueConsumer::acquireBuffer
04-19 19:33:38.926 666 666 E acquireBuffer: #00 pc 0004d34f /system/lib/libgui.so (android::BufferQueueConsumer::acquireBuffer(android::BufferItem*, long long, unsigned long long)+74)
04-19 19:33:38.926 666 666 E acquireBuffer: #01 pc 000645cf /system/lib/libgui.so (android::ConsumerBase::acquireBufferLocked(android::BufferItem*, long long, unsigned long long)+62)
04-19 19:33:38.926 666 666 E acquireBuffer: #02 pc 0007a7a1 /system/lib/libsurfaceflinger.so (android::FramebufferSurface::advanceFrame(bool)+112)
04-19 19:33:38.926 666 666 E acquireBuffer: #03 pc 000edf1f /system/lib/libsurfaceflinger.so (android::compositionengine::impl::RenderSurface::queueBuffer(android::base::unique_fd_impl, bool)+358)
04-19 19:33:38.926 666 666 E acquireBuffer: #04 pc 000e46e7 /system/lib/libsurfaceflinger.so (android::compositionengine::impl::Output::finishFrame(android::compositionengine::CompositionRefreshArgs const&)+454)
04-19 19:33:38.926 666 666 E acquireBuffer: #05 pc 000de3e5 /system/lib/libsurfaceflinger.so (android::compositionengine::impl::Display::finishFrame(android::compositionengine::CompositionRefreshArgs const&)+72)
04-19 19:33:38.926 666 666 E acquireBuffer: #06 pc 000e3011 /system/lib/libsurfaceflinger.so (android::compositionengine::impl::Output::present(android::compositionengine::CompositionRefreshArgs const&)+92)
04-19 19:33:38.926 666 666 E acquireBuffer: #07 pc 000dcfa1 /system/lib/libsurfaceflinger.so (android::compositionengine::impl::CompositionEngine::present(android::compositionengine::CompositionRefreshArgs&)+144)
04-19 19:33:38.926 666 666 E acquireBuffer: #08 pc 000baf81 /system/lib/libsurfaceflinger.so (android::SurfaceFlinger::onMessageRefresh()+1280)
04-19 19:33:38.926 666 666 E acquireBuffer: #09 pc 000b8b1d /system/lib/libsurfaceflinger.so (android::SurfaceFlinger::onMessageReceived(int, long long)+52)
releaseBuffer
方法位置：BufferQueueConsumer::releaseBuffer
04-19 19:33:38.996 666 666 E releaseBuffer: #00 pc 0004e429 /system/lib/libgui.so (android::BufferQueueConsumer::releaseBuffer(int, unsigned long long, android::sp const&, void*, void*)+72)
04-19 19:33:38.996 666 666 E releaseBuffer: #01 pc 0004f3af /system/lib/libgui.so (android::BufferQueueConsumer::releaseBuffer(int, unsigned long long, void*, void*, android::sp const&)+36)
04-19 19:33:38.996 666 666 E releaseBuffer: #02 pc 0006498d /system/lib/libgui.so (android::ConsumerBase::releaseBufferLocked(int, android::sp, void*, void*)+140)
04-19 19:33:38.996 666 666 E releaseBuffer: #03 pc 00066cab /system/lib/libsurfaceflinger.so (android::BufferLayerConsumer::releasePendingBuffer()+78)
04-19 19:33:38.996 666 666 E releaseBuffer: #04 pc 00067b1d /system/lib/libsurfaceflinger.so (android::BufferQueueLayer::releasePendingBuffer(long long)+28)
04-19 19:33:38.996 666 666 E releaseBuffer: #05 pc 000bb0bb /system/lib/libsurfaceflinger.so (android::SurfaceFlinger::onMessageRefresh()+1594)
04-19 19:33:38.996 666 666 E releaseBuffer: #06 pc 000b8b1d /system/lib/libsurfaceflinger.so (android::SurfaceFlinger::onMessageReceived(int, long long)+52)
2.2.3、认识BufferQueueCore
BufferQueueCore是bufferqueue实现的具体类。BufferQueueCore管理了几个数据结构。操作这几个数据结构，来实现bufferqueue的dequebuffer等方法。
以dequeueBuffer过程来讲，它就是向bufferqueue申请一个GraphicBuffer,用GraphicBuffer来绘制图像。
BufferQueueCore中六个对象比较重要：mQueue、mSlot、FreeSlots、FreeBuffers、ActiveBuffers、UnusedBuffers 下面我会一一介绍
namespace android {
class IConsumerListener;
class IProducerListener;
class BufferQueueCore : public virtual RefBase {
private:
BufferQueueDefs::SlotsType mSlots;
Fifo mQueue;
std::set mFreeSlots;
std::list mFreeBuffers;
std::list mUnusedSlots;
std::set mActiveBuffers;
1、mQueue&&mSlot
我们先认识这两个最核心的数据结构。
class BufferItem {
public:
enum { INVALID_BUFFER_SLOT = -1 };
BufferItem() : mGraphicBuffer(nullptr), mFence(Fence::NO_FENCE) {}
~BufferItem() {}
sp mGraphicBuffer;
sp mFence;
Rect mCrop;
typedef Vector Fifo;
Fifo mQueue;
namespace BufferQueueDefs {
typedef BufferSlot SlotsType[NUM_BUFFER_SLOTS];
}
BufferQueueDefs::SlotsType mSlots;
我们平时说的bufferqueue队列其实就是说的BufferItem这个mQueue数据容器。BufferItem拥有GraphicBuffer对象
而mSlot是一个BufferSlot大小为NUM_BUFFER_SLOTS(等于64)的数组，BufferSlot主要用来绑定GraphicBuffer,BufferSlot和GraphicBuffer一一对应。
BufferSlot中有一个BufferState对象，它专门用来表示GraphicBuffer的状态
struct BufferSlot {
BufferState mBufferState;
// BufferState tracks the states in which a buffer slot can be.
struct BufferState {
// All slots are initially FREE (not dequeued, queued, acquired, or shared).
BufferState()
: mDequeueCount(0),
mQueueCount(0),
mAcquireCount(0),
mShared(false) {
}
uint32_t mDequeueCount;
uint32_t mQueueCount;
uint32_t mAcquireCount;
bool mShared;
// A buffer can be in one of five states, represented as below:
//
// | mShared | mDequeueCount | mQueueCount | mAcquireCount |
// --------|---------|---------------|-------------|---------------|
// FREE | false | 0 | 0 | 0 |
// DEQUEUED| false | 1 | 0 | 0 |
// QUEUED | false | 0 | 1 | 0 |
// ACQUIRED| false | 0 | 0 | 1 |
// SHARED | true | any | any | any |
2、FreeSlots、FreeBuffers、ActiveBuffers、UnusedBuffers
知道了mQueue&&mSlot作用后，我们再来认识他们四个容易了,首先成立这样一个等式
mSlots = mFreeSlots + mFreeBuffers + mActiveBuffers + mUnusedSlots
mSlots是所有BufferSlot，而其它四个BufferSlot，则表示不同的状态的BufferSlot。
mFreeSlots：BufferSlot状态为FREE，且没有GraphicBuffer与之相绑定的slot集合
mFreeBuffers：BufferSlot状态为FREE，且有GraphicBuffer与之相绑定的slot集合
mActiveBuffers：BufferSlot状态不为FREE（即DEQUEUED、QUEUED、ACQUIRED、SHARED）的slot集合。既然状态不是FREE，那么该BufferSlot必然有一个GraphicBuffer与之相绑定
mUnusedSlots：未参与使用的slot集合，由 mMaxBufferCount 决定
2.2.4、BufferQueueProducer生产者
生产者这边，在bufferqueue流程中，主要负责dequeueBuffer、queueBuffer等流程
class BufferQueueProducer : public BnGraphicBufferProducer {
public:
friend class BufferQueue; // Needed to access binderDied
explicit BufferQueueProducer(const sp& core,
bool consumerIsSurfaceFlinger = false);
~BufferQueueProducer() override;
virtual status_t requestBuffer(int slot, sp* buf);
virtual status_t dequeueBuffer(int* outSlot, sp* outFence, uint32_t width,
uint32_t height, PixelFormat format, uint64_t usage,
uint64_t* outBufferAge,
FrameEventHistoryDelta* outTimestamps) override;
virtual status_t queueBuffer(int slot,
const QueueBufferInput& input, QueueBufferOutput* output);
...//等方法
1、dequeueBuffer
status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp* outFence,
uint32_t width, uint32_t height, PixelFormat format,
uint64_t usage, uint64_t* outBufferAge,
FrameEventHistoryDelta* outTimestamps) {
// 获取下一个buffer slot,如果有freeSlot直接拿来用
while (found == BufferItem::INVALID_BUFFER_SLOT) {
status_t status = waitForFreeSlotThenRelock(FreeSlotCaller::Dequeue, lock, &found);
if (status != NO_ERROR) {
return status;
}
...
// IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION 和IGraphicBufferProducer::RELEASE_ALL_BUFFERS标记会走 requestBuffer流程来获取graphicBuffer对应的slot
if ((buffer == nullptr) ||
buffer->needsReallocation(width, height, format, BQ_LAYER_COUNT, usage))
{
mSlots[found].mAcquireCalled = false;
mSlots[found].mGraphicBuffer = nullptr;
mSlots[found].mRequestBufferCalled = false;
mSlots[found].mEglDisplay = EGL_NO_DISPLAY;
mSlots[found].mEglFence = EGL_NO_SYNC_KHR;
mSlots[found].mFence = Fence::NO_FENCE;
mCore->mBufferAge = 0;
mCore->mIsAllocating = true;
returnFlags |= BUFFER_NEEDS_REALLOCATION;
}
上面摘抄了dequebuffer流程主要的两件事情
1、获取下一个buffer slot,如果有freeSlot直接拿来用
2、IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION 和IGraphicBufferProducer::RELEASE_ALL_BUFFERS标记会走 requestBuffer流程来获取graphicBuffer对应的slot
2、queueBuffer
queueBuffer将已填充的缓冲区返回给BufferQueue，即把bufferslot封装成BufferItem返回到BufferItem队列mQueue。
status_t BufferQueueProducer::queueBuffer(int slot,
const QueueBufferInput &input, QueueBufferOutput *output) {
// 构建bufferitem并赋值
BufferItem item;
item.mAcquireCalled = mSlots[slot].mAcquireCalled;
item.mGraphicBuffer = mSlots[slot].mGraphicBuffer;
...
item.mSlot = slot;
item.mFence = acquireFence;
item.mFenceTime = acquireFenceTime;
//bufferItem 入队
if (mCore->mQueue.empty()) {
// When the queue is empty, we can ignore mDequeueBufferCannotBlock
// and simply queue this buffer
mCore->mQueue.push_back(item);
frameAvailableListener = mCore->mConsumerListener;
} else {
// When the queue is not empty, we need to look at the last buffer
// in the queue to see if we need to replace it
const BufferItem& last = mCore->mQueue.itemAt(
mCore->mQueue.size() - 1);
if (last.mIsDroppable) {
if (!last.mIsStale) {
mSlots[last.mSlot].mBufferState.freeQueued();
queueBuffer比较简单，主要两步
1、构建bufferItem，赋值等
2、把bufferItem返回到mQueue队列中
2.2.4、BufferQueueConsumer消费者
1、acquireBuffer
acquireBuffer方法从缓冲队列中尝试取下一个挂起的BufferItem有这些情况：
队列为空直接返回；如果缓冲已被获取，返回之前获取的缓冲项；如果expectedPresent标记为非0，缓冲会即将显示；如果缓冲时间戳在将来，不会被获取；
status_t BufferQueueConsumer::acquireBuffer(BufferItem* outBuffer,
nsecs_t expectedPresent, uint64_t maxFrameNumber) {
// 新旧缓冲区显示逻辑，决定显示的buffer
if (expectedPresent != 0 && !mCore->mQueue.empty()) {
while (mCore->mQueue.size() > 1 && !mCore->mQueue[0].mIsAutoTimestamp) {
const BufferItem& bufferItem(mCore->mQueue[1]);
...
此段逻辑比较简单，就是拿缓存，逻辑主要是怎么拿，最后回调
2、releaseBuffer
releaseBuffer将bufferslot返回到bufferqueue队列。可以在bufferqueue任在访问时进行，当buffer不再可用，fence机制会发出信号。
如果releaseBuffer收到STALE_BUFFER_SLOT信号，那么consumer必须直接放弃所有引用。直接释放。
代码就是release为主
status_t BufferQueueConsumer::releaseBuffer(int slot, uint64_t frameNumber,
const sp& releaseFence, EGLDisplay eglDisplay,
EGLSyncKHR eglFence) {
sp listener;
{ // Autolock scope
std::lock_guard lock(mCore->mMutex);
if (frameNumber != mSlots[slot].mFrameNumber &&
!mSlots[slot].mBufferState.isShared()) {
return STALE_BUFFER_SLOT;
}
if (!mSlots[slot].mBufferState.isAcquired()) {
BQ_LOGE("releaseBuffer: attempted to release buffer slot %d "
"but its state was %s", slot,
mSlots[slot].mBufferState.string());
return BAD_VALUE;
}
mSlots[slot].mEglDisplay = eglDisplay;
mSlots[slot].mEglFence = eglFence;
mSlots[slot].mFence = releaseFence;
mSlots[slot].mBufferState.release();
三、bufferqueue总结
bufferqueue是整个安卓图形处理的核心，把数据生产者和数据消费者连接起来。
生产者dequeuebuffer申请一个缓冲区，并指定宽高等。填充缓冲区数据后用queueBuffer返回到队列。随后aquirebuffer获取缓冲区，消费者处理完后。则返回到队列release