上一篇介绍了BufferQueue队列,这篇简单介绍下GraphicBuffer在BufferQueue队列中的传递过程。
先看下从生产者Surface到消费者Layer之间的结构图:
Surface: 持有了Buffer队列的生产者代理端,可以获取新的Buffer来填充内容,然后再放入Buffer队列
Buffer队列:Buffer队列的生产者端被Surface持有,消费者端被SurfaceFlingerConsumer持有,SurfaceFlingerConsumer: 继承GLConsumer和ConsumerBase,对BufferQueue的消费者做了封装用于方便SurfaceFlinger处理图像数据。
Layer: 当SurfaceFlinger接收到生产者传递过来的数据后会转发给Layer进行处理。
在进行图像传递流程之前,生产者和消费者必须connect到BufferQueue。
Surfae绘制送显流程
suface.lock();
//drawUI
surface.unlockAndPost();
一般应用要绘制UI遵循以上步骤,先执行Surface的lock方法,然后进行UI绘制,绘制完成后调用Surface的unlockAndPost方法交由系统显示。
status_t Surface::lock(
ANativeWindow_Buffer* outBuffer, ARect* inOutDirtyBounds)
{
//从BufferQueue队列申请一个Buffer
ANativeWindowBuffer* out;
int fenceFd = -1;
status_t err = dequeueBuffer(&out, &fenceFd);
if (err == NO_ERROR) {
sp backBuffer(GraphicBuffer::getSelf(out));
...
//对该GraphicBuffer上锁
void* vaddr;
status_t res = backBuffer->lockAsync(
GRALLOC_USAGE_SW_READ_OFTEN | GRALLOC_USAGE_SW_WRITE_OFTEN,
newDirtyRegion.bounds(), &vaddr, fenceFd);
//将Buffer传递給outBuffer,由其他部分对outBuffer进行绘制
if (res != 0) {
err = INVALID_OPERATION;
} else {
mLockedBuffer = backBuffer;
outBuffer->width = backBuffer->width;
outBuffer->height = backBuffer->height;
outBuffer->stride = backBuffer->stride;
outBuffer->format = backBuffer->format;
outBuffer->bits = vaddr;
}
}
return err;
}
Surface Lock方法主要实现了两个内容:
1:调用dequeueBuffer从队列获取新的GraphicBuffer
2:调用GraphicBuffer的lockAsync对Buffer进行上锁,应用就可以对该Buffer进行内容绘制了。
status_t Surface::unlockAndPost()
{
//对dequeue的buffer解锁
int fd = -1;
status_t err = mLockedBuffer->unlockAsync(&fd);
ALOGE_IF(err, "failed unlocking buffer (%p)", mLockedBuffer->handle);
//将绘制完内容的buffer调用queueBuffer放入队列
err = queueBuffer(mLockedBuffer.get(), fd);
....
return err;
}
Surface的unlockAndPost方法也实现了两个内容:
1:对上一步上锁的Buffer解锁
2:将绘制完成的GraphicBuffer放入队列,准备显示
Surface 获取Buffer
Surface.dequeueBuffer
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {
....
int buf = -1;
sp fence;
//从BufferQueue中BufferSlot中申请一个Slot用来填充内容
status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence,
reqWidth, reqHeight, reqFormat, reqUsage);
Mutex::Autolock lock(mMutex);
//根据从BufferQueue申请到的slot值buf, 在surface自己的mSlots中数组中获取mSlot对应的GraphicBuffer指针
sp& gbuf(mSlots[buf].buffer);
//如果Buffer属性发生了变化,原来的Buffer不在适用, Surface自己的mSlots数组需要释放掉对应的GraphicBuffer
if (result & IGraphicBufferProducer::RELEASE_ALL_BUFFERS) {
freeAllBuffers();
}
//如果Surface自己的mSlots数组中的对应的序号的slot中还没有GraphicBuffer, 则调用requestBuffer从BufferQueue对应的
//slot中将GraphicBuffer的指针拿到,保存到surface的mSlots中
if ((result & IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION) || gbuf == 0) {
result = mGraphicBufferProducer->requestBuffer(buf, &gbuf);
if (result != NO_ERROR) {
ALOGE("dequeueBuffer: IGraphicBufferProducer::requestBuffer failed: %d", result);
mGraphicBufferProducer->cancelBuffer(buf, fence);
return result;
}
}
....
//将GraphicBuffer指针保存到出参buffer中
*buffer = gbuf.get();
return OK;
}
Surface中有一个数组变量mSlots,和BufferQueueCore中的mSlots不太一样,太用于保存BufferQueue中对应slot位置的GraphicBuffer指针。每次dequeueBuffer的时候,会先根据index值看自己的mSlot数组对应位置是否已经保存了GraphicBuffer指针,有的话如果条件符合就直接返回,否则就需要从BufferQueue中申请获取对应slot的图像缓冲区指针。这样不用每次都从BufferQueue中获取,节省时间和资源。
BufferQueueProducer.dequeueBuffer
status_t BufferQueueProducer::dequeueBuffer(int *outSlot,
sp *outFence, uint32_t width, uint32_t height,
PixelFormat format, uint32_t usage) {
...
{ // Autolock scope
//循环查找符合条件的slot
int found = BufferItem::INVALID_BUFFER_SLOT;
while (found == BufferItem::INVALID_BUFFER_SLOT) {
status_t status = waitForFreeSlotThenRelock(FreeSlotCaller::Dequeue,
&found);
......
const sp& buffer(mSlots[found].mGraphicBuffer);
if (!mCore->mAllowAllocation) {
if (buffer->needsReallocation(width, height, format, usage)) {
...
//如果允许Producer自行分配新的GraphicBuffer的情况下
//GraphicBuffer发生变化需要重新申请,则把对应的slot放入freeSlot数组中
mCore->mFreeSlots.insert(found);
mCore->clearBufferSlotLocked(found);
found = BufferItem::INVALID_BUFFER_SLOT;
continue;
}
}
}
const sp& buffer(mSlots[found].mGraphicBuffer);
//将查找的slot值保存到出参。
*outSlot = found;
//将slot对应BufferState的状态设置为DEQUEUE
mSlots[found].mBufferState.dequeue();
//如果对应slot中的BufferSlot中尚且没有分配GraphicBuffer,或者GraphicBuffer需要重新申请
//则重置该slot对应的参数, 将flag设置为BUFFER_NEEDS_REALLOCATION
if ((buffer == NULL) ||
buffer->needsReallocation(width, height, format, usage))
{
mSlots[found].mAcquireCalled = false;
mSlots[found].mGraphicBuffer = NULL;
mSlots[found].mRequestBufferCalled = false;
mSlots[found].mEglDisplay = EGL_NO_DISPLAY;
mSlots[found].mEglFence = EGL_NO_SYNC_KHR;
mSlots[found].mFence = Fence::NO_FENCE;
mCore->mBufferAge = 0;
mCore->mIsAllocating = true;
returnFlags |= BUFFER_NEEDS_REALLOCATION;
}
...
} // Autolock scope
//为slot分配GraphicBuffer
if (returnFlags & BUFFER_NEEDS_REALLOCATION) {
//调用Gralloc模块分配新的GraphicBuffer
sp graphicBuffer(mCore->mAllocator->createGraphicBuffer(
width, height, format, usage, &error));
{ // Autolock scope
Mutex::Autolock lock(mCore->mMutex);
//保存在mSlots数组对应的BufferSlot中
if (graphicBuffer != NULL && !mCore->mIsAbandoned) {
graphicBuffer->setGenerationNumber(mCore->mGenerationNumber);
mSlots[*outSlot].mGraphicBuffer = graphicBuffer;
}
...
} // Autolock scope
}
if (attachedByConsumer) {
returnFlags |= BUFFER_NEEDS_REALLOCATION;
}
...
//调用者surface,会根据flag判断自己持有的GraphicBuffer是否失效,失效的话需要重新申请
return returnFlags;
}
对该函数做了精简,移除了不太关注的部分。
该函数关于dequeueBuffer的内容工作主要如下
1:从BufferQueue中查找符合的slot值,while循环,直到找到合适的slot为止
2:将slot返回个调用者
3:图像缓冲区尚未分配或者需要重新分配则重置参数,调用Gralloc分配新的GraphicBuffer
下面再简单看下查找slot的过程waitForFreeSlotThenRelock函数
status_t BufferQueueProducer::waitForFreeSlotThenRelock(FreeSlotCaller caller,
int* found) const {
//是否需要重新尝试查找
bool tryAgain = true;
while (tryAgain) {
if (mCore->mIsAbandoned) {
//BufferQueueCore 已经失效,直接返回
return NO_INIT;
}
//统计dequeuBuffer和acquireBuffer的个数
int dequeuedCount = 0;
int acquiredCount = 0;
for (int s : mCore->mActiveBuffers) {
if (mSlots[s].mBufferState.isDequeued()) {
++dequeuedCount;
}
if (mSlots[s].mBufferState.isAcquired()) {
++acquiredCount;
}
}
// 简述生产者是否申请了过多的Buffer
if (mCore->mBufferHasBeenQueued &&
dequeuedCount >= mCore->mMaxDequeuedBufferCount) {
BQ_LOGE("%s: attempting to exceed the max dequeued buffer count "
"(%d)", callerString, mCore->mMaxDequeuedBufferCount);
return INVALID_OPERATION;
}
*found = BufferQueueCore::INVALID_BUFFER_SLOT;
const int maxBufferCount = mCore->getMaxBufferCountLocked();
bool tooManyBuffers = mCore->mQueue.size()
> static_cast(maxBufferCount);
if (tooManyBuffers) {
BQ_LOGV("%s: queue size is %zu, waiting", callerString,
mCore->mQueue.size());
} else {
......
if (caller == FreeSlotCaller::Dequeue) {
// 首先尝试从FreeBuffer列表中获取slot
int slot = getFreeBufferLocked();
if (slot != BufferQueueCore::INVALID_BUFFER_SLOT) {
*found = slot;
} else if (mCore->mAllowAllocation) {
//从FreeSlot列表中获取slot
*found = getFreeSlotLocked();
}
}
}
// 如果没有找到合适的BufferSlot,或者已经申请了过多的Buffer,则需要等待有释放的Buffer后重新申请
tryAgain = (*found == BufferQueueCore::INVALID_BUFFER_SLOT) ||
tooManyBuffers;
if (tryAgain) {
//如果不能阻塞的话则直接返回
if ((mCore->mDequeueBufferCannotBlock || mCore->mAsyncMode) &&
(acquiredCount <= mCore->mMaxAcquiredBufferCount)) {
return WOULD_BLOCK;
}
//否则,等待有释放新的Buffer后再申请
if (mDequeueTimeout >= 0) {
status_t result = mCore->mDequeueCondition.waitRelative(
mCore->mMutex, mDequeueTimeout);
if (result == TIMED_OUT) {
return result;
}
} else {
mCore->mDequeueCondition.wait(mCore->mMutex);
}
}
} // while (tryAgain)
return NO_ERROR;
}
查找主要流程如下:
1:首先统计处于dequeueB状态和acquire状态的Buffer的个数
2:如果生产者已经申请了Buffer,且申请的个数大于最大可申请个数,则不允许申请
3:判断BufferQueue中Buffer个数是否达到最大限度
4:生产者申请Buffer,优先从freeBuffer列表(slot处于FREE状态,且已经分配了Buffer)获取,然后再从freeSlot(处于slot状态还未分配Buffer)列表获取
5:如果仍然没有找到,则等待有Buffer释放后再次尝试申请
此时,Surface 获取Buffer的流程已经完成了,Surface已经获取到了一个可用的GraphicBuffer
Surface 将Buffer放入队列
Surface.queueBuffer
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
Mutex::Autolock lock(mMutex);
int64_t timestamp;
bool isAutoTimestamp = false;
//记录QueueBuffer的时间
if (mTimestamp == NATIVE_WINDOW_TIMESTAMP_AUTO) {
timestamp = systemTime(SYSTEM_TIME_MONOTONIC);
isAutoTimestamp = true;
ALOGV("Surface::queueBuffer making up timestamp: %.2f ms",
timestamp / 1000000.f);
} else {
timestamp = mTimestamp;
}
//根据Buffer从Surface的mSlots数组中找到对应的slot值
int i = getSlotFromBufferLocked(buffer);
//初始化queueBuffer 的入参input,和出参output
sp fence(fenceFd >= 0 ? new Fence(fenceFd) : Fence::NO_FENCE);
IGraphicBufferProducer::QueueBufferOutput output;
IGraphicBufferProducer::QueueBufferInput input(timestamp, isAutoTimestamp,
mDataSpace, crop, mScalingMode, mTransform ^ mStickyTransform,
fence, mStickyTransform);
//判断是否需要做屏幕旋转,如果需要旋转,则将宽,高互换一下
int width = buffer->width;
int height = buffer->height;
bool rotated90 = (mTransform ^ mStickyTransform) &
NATIVE_WINDOW_TRANSFORM_ROT_90;
if (rotated90) {
std::swap(width, height);
}
//判断所有的dirtyRegion, 如果需要旋转则做一次旋转
//最终的变化区域存放在了flippedRegion变量中
Region flippedRegion;
for (auto rect : mDirtyRegion) {
int left = rect.left;
int right = rect.right;
int top = height - rect.bottom; // Flip from OpenGL convention
int bottom = height - rect.top; // Flip from OpenGL convention
switch (mTransform ^ mStickyTransform) {
case NATIVE_WINDOW_TRANSFORM_ROT_90: {
// Rotate 270 degrees
Rect flippedRect{top, width - right, bottom, width - left};
flippedRegion.orSelf(flippedRect);
break;
}
case NATIVE_WINDOW_TRANSFORM_ROT_180: {
// Rotate 180 degrees
Rect flippedRect{width - right, height - bottom,
width - left, height - top};
flippedRegion.orSelf(flippedRect);
break;
}
case NATIVE_WINDOW_TRANSFORM_ROT_270: {
// Rotate 90 degrees
Rect flippedRect{height - bottom, left,
height - top, right};
flippedRegion.orSelf(flippedRect);
break;
}
default: {
Rect flippedRect{left, top, right, bottom};
flippedRegion.orSelf(flippedRect);
break;
}
}
}
//将变换区域存放在input参数中
input.setSurfaceDamage(flippedRegion);
//调用GraphicBufferProducer的queueBuffer将并Buffer slot值和input作为入参。此函数将Buffer放入队列
status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);
//获取queueBuffer返回的信息
output.deflate(&mDefaultWidth, &mDefaultHeight, &hint,
&numPendingBuffers);
......
return err;
}
Surface的queueBuffer做了什么工作呢?
1:根据填充完的Buffer,找到该Buffer对应的slot值。
2:初始化时间,以及queueBuffer函数的入参input和出参output。
3:判断是否需要进行屏幕旋转,将所以的变化的区域都做对应的旋转操作
4:调用queueBuffer将slot对应的GraphicBuffer放入队列
5:根据返回的output做部分处理
BufferQueueProducer.queueBuffer
status_t BufferQueueProducer::queueBuffer(int slot,
const QueueBufferInput &input, QueueBufferOutput *output) {
int64_t timestamp;
bool isAutoTimestamp;
android_dataspace dataSpace;
Rect crop(Rect::EMPTY_RECT);
int scalingMode;
uint32_t transform;
uint32_t stickyTransform;
sp fence;
//解析input相关的参数
input.deflate(×tamp, &isAutoTimestamp, &dataSpace, &crop, &scalingMode,
&transform, &fence, &stickyTransform);
Region surfaceDamage = input.getSurfaceDamage();
sp frameAvailableListener;
sp frameReplacedListener;
int callbackTicket = 0;
BufferItem item;
{ // Autolock scope
Mutex::Autolock lock(mCore->mMutex);
//做简单的校验
//BufferQueue是否处于Abandon状态
if (mCore->mIsAbandoned) {
BQ_LOGE("queueBuffer: BufferQueue has been abandoned");
return NO_INIT;
}
//生产者时候连接到了BufferQueue队列
if (mCore->mConnectedApi == BufferQueueCore::NO_CONNECTED_API) {
BQ_LOGE("queueBuffer: BufferQueue has no connected producer");
return NO_INIT;
}
//slot值是否有效
if (slot < 0 || slot >= BufferQueueDefs::NUM_BUFFER_SLOTS) {
BQ_LOGE("queueBuffer: slot index %d out of range [0, %d)",
slot, BufferQueueDefs::NUM_BUFFER_SLOTS);
return BAD_VALUE;
//slot对应的BufferState是否处于Dequeue状态,不是Dequeue状态的buffer不能跳转到Queue状态
} else if (!mSlots[slot].mBufferState.isDequeued()) {
BQ_LOGE("queueBuffer: slot %d is not owned by the producer "
"(state = %s)", slot, mSlots[slot].mBufferState.string());
return BAD_VALUE;
//slot对应的Buffer是否被Surface request过
} else if (!mSlots[slot].mRequestBufferCalled) {
BQ_LOGE("queueBuffer: slot %d was queued without requesting "
"a buffer", slot);
return BAD_VALUE;
}
const sp& graphicBuffer(mSlots[slot].mGraphicBuffer);
Rect bufferRect(graphicBuffer->getWidth(), graphicBuffer->getHeight());
Rect croppedRect(Rect::EMPTY_RECT);
crop.intersect(bufferRect, &croppedRect);
if (croppedRect != crop) {
BQ_LOGE("queueBuffer: crop rect is not contained within the "
"buffer in slot %d", slot);
return BAD_VALUE;
}
// Override UNKNOWN dataspace with consumer default
if (dataSpace == HAL_DATASPACE_UNKNOWN) {
dataSpace = mCore->mDefaultBufferDataSpace;
}
mSlots[slot].mFence = fence;
//将BufferState状态修改为QUEUED状态
mSlots[slot].mBufferState.queue();
//记录当前Buffer的FrameNumer序号,每次来一帧信号则+1
++mCore->mFrameCounter;
mSlots[slot].mFrameNumber = mCore->mFrameCounter;
//设置BufferItem对象的属性,将当前slot对应的信息保存到BufferItem对象中
item.mAcquireCalled = mSlots[slot].mAcquireCalled;
item.mGraphicBuffer = mSlots[slot].mGraphicBuffer;
item.mCrop = crop;
item.mTransform = transform &
~static_cast(NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY);
item.mTransformToDisplayInverse =
(transform & NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY) != 0;
item.mScalingMode = static_cast(scalingMode); //缩放模式
item.mTimestamp = timestamp; //时间戳
item.mIsAutoTimestamp = isAutoTimestamp; //是否是自动生成的时间
item.mDataSpace = dataSpace; //dataSpace熟悉
item.mFrameNumber = mCore->mFrameCounter;
item.mSlot = slot;//当前GraphicBuffer的slot值
item.mFence = fence; //当前GraphicBuffer的fence值
item.mIsDroppable = mCore->mAsyncMode ||
mCore->mDequeueBufferCannotBlock ||
(mCore->mSharedBufferMode && mCore->mSharedBufferSlot == slot);
item.mSurfaceDamage = surfaceDamage; //保存当前GraphicBuffer中内容变化区域
item.mQueuedBuffer = true; //Item是否是Queue状态
item.mAutoRefresh = mCore->mSharedBufferMode && mCore->mAutoRefresh;
mStickyTransform = stickyTransform;
//BufferItem用来描述一个GraphicBuffer相关的内容,从生产者发送过来的Buffer都会保存在BufferQueue中的一个mQueue列表中。
if (mCore->mQueue.empty()) {
//如果放入之前,mQueue队列为空,则直接放入即可
mCore->mQueue.push_back(item);
frameAvailableListener = mCore->mConsumerListener;
} else {
//如果放入新的Buffer之前,队列中还有等待消费者处理显示的Buffer,说明消费这处理速度较慢
const BufferItem& last = mCore->mQueue.itemAt(
mCore->mQueue.size() - 1);
if (last.mIsDroppable) {
//如果上一帧的Buffer的mIsDroppable为true, 说明该Buffer可被放弃,直接在队列中将新的Buffer替换旧的Buffer
mCore->mQueue.editItemAt(mCore->mQueue.size() - 1) = item;
frameReplacedListener = mCore->mConsumerListener;
} else {
//否则将新的Buffer放到队列的末尾
mCore->mQueue.push_back(item);
frameAvailableListener = mCore->mConsumerListener;
}
}
//设置BufferQueue的mBufferHasBeenQueued值为true,表示有Buffer成功入队列,等待消费
mCore->mBufferHasBeenQueued = true;
mCore->mDequeueCondition.broadcast();
mCore->mLastQueuedSlot = slot;
//设置output的属性,返回给调用者
output->inflate(mCore->mDefaultWidth, mCore->mDefaultHeight,
mCore->mTransformHint,
static_cast(mCore->mQueue.size()));
ATRACE_INT(mCore->mConsumerName.string(), mCore->mQueue.size());
// Take a ticket for the callback functions
callbackTicket = mNextCallbackTicket++;
VALIDATE_CONSISTENCY();
} // Autolock scope
//将BufferItem的GraphicBuffer和slot值置为空,因为消费这不需要这些信息
item.mGraphicBuffer.clear();
item.mSlot = BufferItem::INVALID_BUFFER_SLOT;
//通知消费者有新的Buffer可用
{
Mutex::Autolock lock(mCallbackMutex);
while (callbackTicket != mCurrentCallbackTicket) {
mCallbackCondition.wait(mCallbackMutex);
}
if (frameAvailableListener != NULL) {
frameAvailableListener->onFrameAvailable(item);
} else if (frameReplacedListener != NULL) {
frameReplacedListener->onFrameReplaced(item);
}
++mCurrentCallbackTicket;
mCallbackCondition.broadcast();
}
//保存此次Buffer相关的内容
mLastQueueBufferFence = fence;
mLastQueuedCrop = item.mCrop;
mLastQueuedTransform = item.mTransform;
return NO_ERROR;
}
此方法还是比较长的,但是逻辑却不是很复杂,代码里已经有详细注释,只简单罗列下步骤
1:解析input相关的参数
2:做一些简单的校验,确保生产者,slot等参数合法
3:修改BufferState状态为QUEUED,且记录FrameNumer值
4:根据Slot的Buffer信息,初始化BufferItem对象,并将item放入BufferQueue的mQueue队列中
放入前队列为空,直接放入即可
放入前队列中还有未来得及处理的Buffer, 如果可以放弃,则直接将Buffer替换, 否则放入队列末尾
5:通知消费者有新的Buffer可用
onFrameAvailable会通知消费者有新的Buffer到来,这个会调用到ConsumerBase的onFrameAvaliable函数中, 而ConsumerBase又会通知Layer来进行处理,直接看下Layer的onFrameAvaliable
void Layer::onFrameAvailable(const BufferItem& item) {
{ // Autolock scope
Mutex::Autolock lock(mQueueItemLock);
//确保上一帧和这一帧是按顺序处理的,否则就需要等待
while (item.mFrameNumber != mLastFrameNumberReceived + 1) {
status_t result = mQueueItemCondition.waitRelative(mQueueItemLock,
ms2ns(500));
if (result != NO_ERROR) {
ALOGE("[%s] Timed out waiting on callback", mName.string());
}
}
//将待处理的bufferItem,放入Layer自己带mQueueItems队列中。然后将mQueuedFrames计数+1,
//表示Layer中有几个BufferItem待处理
mQueueItems.push_back(item);
android_atomic_inc(&mQueuedFrames);
//保存这一帧的FrameNumer计数
mLastFrameNumberReceived = item.mFrameNumber;
mQueueItemCondition.broadcast();
}
//通知SurfaceFlinger更新,进行图像合成
mFlinger->signalLayerUpdate();
}
Layer接收到新Buffer的通知后,将bufferItem保存到自己的mQueueItems队列中,计数+1, 然后通知SurfaceFlinger进行更新,重新合成图像。