上一篇文章分析了dequeueBuffer
函数的过程,本篇接着分析queueBuffer
函数,当我们需要绘制图像时,调用dequeueBuffer
函数获取到可用的GraphicBuffer
之后就可以开始绘制了,最常见的绘制操作就是Android上层View的draw
方法了,其他还有OpenGL ES、 mediaserver 视频解码器都可以作为图形数据的来源。
当对GraphicBuffer
的绘制操作完成之后就需要调用queueBuffer
函数将这块buffer放入BufferQueue
队列中并通过回调通知消费者使用这块buffer,图形数据最常见的消耗方就是SurfaceFlinger
,该系统服务会消耗当前可见的 Surface
(核心是GraphicBuffer
),合成到显示部分。
具体实的合成流程是SurfaceFlinger
使用 OpenGL
和Hardware Composer来合成一组 Surface。
其他 OpenGL ES 应用也可以消耗图像流,例如相机应用会消耗相机预览图像流。非 GL 应用也可以是使用方,例如 ImageReader 类。(引自Android官网)。
我们从Surface.cpp
的queueBuffer
函数开始看,这个函数比较长我们分为三部分来看。
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
......
int i = getSlotFromBufferLocked(buffer);
if (i < 0) {
if (fenceFd >= 0) {
close(fenceFd);
}
return i;
}
if (mSharedBufferSlot == i && mSharedBufferHasBeenQueued) {
if (fenceFd >= 0) {
close(fenceFd);
}
return OK;
}
// Make sure the crop rectangle is entirely inside the buffer.
Rect crop(Rect::EMPTY_RECT);
mCrop.intersect(Rect(buffer->width, buffer->height), &crop);
sp<Fence> fence(fenceFd >= 0 ? new Fence(fenceFd) : Fence::NO_FENCE);
IGraphicBufferProducer::QueueBufferOutput output;
IGraphicBufferProducer::QueueBufferInput input(timestamp, isAutoTimestamp,
static_cast<android_dataspace>(mDataSpace), crop, mScalingMode,
mTransform ^ mStickyTransform, fence, mStickyTransform,
mEnableFrameTimestamps);
// we should send HDR metadata as needed if this becomes a bottleneck
input.setHdrMetadata(mHdrMetadata);
ALOGE("dongjiao...mConnectedToCpu = :%d,INVALID_RECT = :%d",mConnectedToCpu,(mDirtyRegion.bounds() == Rect::INVALID_RECT));
if (mConnectedToCpu || mDirtyRegion.bounds() == Rect::INVALID_RECT) {
input.setSurfaceDamage(Region::INVALID_REGION);
} else {
// Here we do two things:
// 1) The surface damage was specified using the OpenGL ES convention of
// the origin being in the bottom-left corner. Here we flip to the
// convention that the rest of the system uses (top-left corner) by
// subtracting all top/bottom coordinates from the buffer height.
// 2) If the buffer is coming in rotated (for example, because the EGL
// implementation is reacting to the transform hint coming back from
// SurfaceFlinger), the surface damage needs to be rotated the
// opposite direction, since it was generated assuming an unrotated
// buffer (the app doesn't know that the EGL implementation is
// reacting to the transform hint behind its back). The
// transformations in the switch statement below apply those
// complementary rotations (e.g., if 90 degrees, rotate 270 degrees).
int width = buffer->width;
int height = buffer->height;
bool rotated90 = (mTransform ^ mStickyTransform) &
NATIVE_WINDOW_TRANSFORM_ROT_90;
if (rotated90) {
std::swap(width, height);
}
Region flippedRegion;
for (auto rect : mDirtyRegion) {
int left = rect.left;
int right = rect.right;
int top = height - rect.bottom; // Flip from OpenGL convention
int bottom = height - rect.top; // Flip from OpenGL convention
switch (mTransform ^ mStickyTransform) {
case NATIVE_WINDOW_TRANSFORM_ROT_90: {
// Rotate 270 degrees
Rect flippedRect{top, width - right, bottom, width - left};
flippedRegion.orSelf(flippedRect);
break;
}
case NATIVE_WINDOW_TRANSFORM_ROT_180: {
// Rotate 180 degrees
Rect flippedRect{width - right, height - bottom,
width - left, height - top};
flippedRegion.orSelf(flippedRect);
break;
}
case NATIVE_WINDOW_TRANSFORM_ROT_270: {
// Rotate 90 degrees
Rect flippedRect{height - bottom, left,
height - top, right};
flippedRegion.orSelf(flippedRect);
break;
}
default: {
Rect flippedRect{left, top, right, bottom};
flippedRegion.orSelf(flippedRect);
break;
}
}
}
input.setSurfaceDamage(flippedRegion);
}
.....
}
首先第一部分,queueBuffer
的第一个参数buffer就是当前绘制完成的GraphicBuffer
,第二个参数fenceFd
是一种用作同步的工具。
getSlotFromBufferLocked
:int Surface::getSlotFromBufferLocked(
android_native_buffer_t* buffer) const {
for (int i = 0; i < NUM_BUFFER_SLOTS; i++) {
if (mSlots[i].buffer != nullptr &&
mSlots[i].buffer->handle == buffer->handle) {
return i;
}
}
ALOGE("getSlotFromBufferLocked: unknown buffer: %p", buffer->handle);
return BAD_VALUE;
}
这个函数很简单,就是找到buffer在mSlots
中的下标,mSlots
为所有BufferSlot
集合,得到下标之后进行一些合法性判断,如果是mSharedBufferSlot
并且mSharedBufferHasBeenQueued
代码当前使用的是共享buffer模式则直接返回OK,不太了解共享buffer模式,就跳过了。
Rect crop
矩形,mCrop
也是一个Rect
对象,用于buffer的裁剪,我们看它的intersect
函数:bool Rect::intersect(const Rect& with, Rect* result) const {
result->left = max(left, with.left);
result->top = max(top, with.top);
result->right = min(right, with.right);
result->bottom = min(bottom, with.bottom);
return !(result->isEmpty());
}
这个函数用于获取两个矩形相交的部分,并将相交部分赋值给result
,再看看调用intersect
传递的参数:
mCrop.intersect(Rect(buffer->width, buffer->height), &crop);
其实就是将mCrop
和Rect(buffer->width, buffer->height)
相交部分赋值给crop
。
接着创建了两个QueueBufferOutput
对象,input
作为输入参数传递到BufferQueueProducer
,input
里面封装的这些信息很多都是外部应用程序设置的,如mDataSpace
,mScalingMode
,mStickyTransform
这些都会应用到最终的显示,而output
会作为BufferQueueProducer
的返回值。
接着有一个判断,mConnectedToCpu
的含义是:CPU生产者(在CPU上运行并生成图形数据的代码,如上层的2Dcanvas
绘图)连接到BufferQueue,可以通过函数native_window_api_connect
传入参数NATIVE_WINDOW_API_CPU
设置CPU生产者,但现在所用的图形数据更多来自OpenGL ES、 mediaserver 视频解码器和camera API,对应连接的API为NATIVE_WINDOW_API_EGL
,NATIVE_WINDOW_API_MEDIA
,
NATIVE_WINDOW_API_CAMERA
,所以mConnectedToCpu
大多数为false。
mDirtyRegion
:脏区域,需要更新图形数据的区域,如果没有需要更新的脏区域则设置无效脏区域。
否则就根据buffer宽高以及旋转角度按一定规则创建flippedRegion
,并设置给input
传递到BufferQueueProducer
。
setSurfaceDamage
设置脏区域,对于非脏区域,会直接从之前的buffer中复制,仅仅更新指定的脏区域部分,提高效率。
我们可以看到所有的图形数据都被封装在了input
中。
Surface::queueBuffer
第一部分分析完了,接着看第二部分:
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
......
status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);
......
return err;
}
这部分就是本篇的核心,调用生产者queueBuffer
函数,传递的依然是BufferSlot
下标,到了BufferQueueProducer
中通过下标就能找到绑定的GraphicBuffer
。
status_t BufferQueueProducer::queueBuffer(int slot,
const QueueBufferInput &input, QueueBufferOutput *output) {
ATRACE_CALL();
ATRACE_BUFFER_INDEX(slot);
int64_t requestedPresentTimestamp;
bool isAutoTimestamp;
android_dataspace dataSpace;
Rect crop(Rect::EMPTY_RECT);
int scalingMode;
uint32_t transform;
uint32_t stickyTransform;
sp<Fence> acquireFence;
bool getFrameTimestamps = false;
input.deflate(&requestedPresentTimestamp, &isAutoTimestamp, &dataSpace,
&crop, &scalingMode, &transform, &acquireFence, &stickyTransform,
&getFrameTimestamps);
const Region& surfaceDamage = input.getSurfaceDamage();
const HdrMetadata& hdrMetadata = input.getHdrMetadata();
if (acquireFence == nullptr) {
BQ_LOGE("queueBuffer: fence is NULL");
return BAD_VALUE;
}
auto acquireFenceTime = std::make_shared<FenceTime>(acquireFence);
switch (scalingMode) {
case NATIVE_WINDOW_SCALING_MODE_FREEZE:
case NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW:
case NATIVE_WINDOW_SCALING_MODE_SCALE_CROP:
case NATIVE_WINDOW_SCALING_MODE_NO_SCALE_CROP:
break;
default:
BQ_LOGE("queueBuffer: unknown scaling mode %d", scalingMode);
return BAD_VALUE;
}
sp<IConsumerListener> frameAvailableListener;
sp<IConsumerListener> frameReplacedListener;
int callbackTicket = 0;
uint64_t currentFrameNumber = 0;
BufferItem item;
{ // Autolock scope
std::lock_guard<std::mutex> lock(mCore->mMutex);
if (mCore->mIsAbandoned) {
BQ_LOGE("queueBuffer: BufferQueue has been abandoned");
return NO_INIT;
}
if (mCore->mConnectedApi == BufferQueueCore::NO_CONNECTED_API) {
BQ_LOGE("queueBuffer: BufferQueue has no connected producer");
return NO_INIT;
}
if (slot < 0 || slot >= BufferQueueDefs::NUM_BUFFER_SLOTS) {
BQ_LOGE("queueBuffer: slot index %d out of range [0, %d)",
slot, BufferQueueDefs::NUM_BUFFER_SLOTS);
return BAD_VALUE;
} else if (!mSlots[slot].mBufferState.isDequeued()) {
BQ_LOGE("queueBuffer: slot %d is not owned by the producer "
"(state = %s)", slot, mSlots[slot].mBufferState.string());
return BAD_VALUE;
} else if (!mSlots[slot].mRequestBufferCalled) {
BQ_LOGE("queueBuffer: slot %d was queued without requesting "
"a buffer", slot);
return BAD_VALUE;
}
// If shared buffer mode has just been enabled, cache the slot of the
// first buffer that is queued and mark it as the shared buffer.
if (mCore->mSharedBufferMode && mCore->mSharedBufferSlot ==
BufferQueueCore::INVALID_BUFFER_SLOT) {
mCore->mSharedBufferSlot = slot;
mSlots[slot].mBufferState.mShared = true;
}
BQ_LOGV("queueBuffer: slot=%d/%" PRIu64 " time=%" PRIu64 " dataSpace=%d"
" validHdrMetadataTypes=0x%x crop=[%d,%d,%d,%d] transform=%#x scale=%s",
slot, mCore->mFrameCounter + 1, requestedPresentTimestamp, dataSpace,
hdrMetadata.validTypes, crop.left, crop.top, crop.right, crop.bottom,
transform,
BufferItem::scalingModeName(static_cast<uint32_t>(scalingMode)));
const sp<GraphicBuffer>& graphicBuffer(mSlots[slot].mGraphicBuffer);
Rect bufferRect(graphicBuffer->getWidth(), graphicBuffer->getHeight());
Rect croppedRect(Rect::EMPTY_RECT);
crop.intersect(bufferRect, &croppedRect);
if (croppedRect != crop) {
BQ_LOGE("queueBuffer: crop rect is not contained within the "
"buffer in slot %d", slot);
return BAD_VALUE;
}
// Override UNKNOWN dataspace with consumer default
if (dataSpace == HAL_DATASPACE_UNKNOWN) {
dataSpace = mCore->mDefaultBufferDataSpace;
}
mSlots[slot].mFence = acquireFence;
mSlots[slot].mBufferState.queue();
// Increment the frame counter and store a local version of it
// for use outside the lock on mCore->mMutex.
++mCore->mFrameCounter;
currentFrameNumber = mCore->mFrameCounter;
mSlots[slot].mFrameNumber = currentFrameNumber;
item.mAcquireCalled = mSlots[slot].mAcquireCalled;
item.mGraphicBuffer = mSlots[slot].mGraphicBuffer;
item.mCrop = crop;
item.mTransform = transform &
~static_cast<uint32_t>(NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY);
item.mTransformToDisplayInverse =
(transform & NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY) != 0;
item.mScalingMode = static_cast<uint32_t>(scalingMode);
item.mTimestamp = requestedPresentTimestamp;
item.mIsAutoTimestamp = isAutoTimestamp;
item.mDataSpace = dataSpace;
item.mHdrMetadata = hdrMetadata;
item.mFrameNumber = currentFrameNumber;
item.mSlot = slot;
item.mFence = acquireFence;
item.mFenceTime = acquireFenceTime;
item.mIsDroppable = mCore->mAsyncMode ||
(mConsumerIsSurfaceFlinger && mCore->mQueueBufferCanDrop) ||
(mCore->mLegacyBufferDrop && mCore->mQueueBufferCanDrop) ||
(mCore->mSharedBufferMode && mCore->mSharedBufferSlot == slot);
item.mSurfaceDamage = surfaceDamage;
item.mQueuedBuffer = true;
item.mAutoRefresh = mCore->mSharedBufferMode && mCore->mAutoRefresh;
item.mApi = mCore->mConnectedApi;
mStickyTransform = stickyTransform;
// Cache the shared buffer data so that the BufferItem can be recreated.
if (mCore->mSharedBufferMode) {
mCore->mSharedBufferCache.crop = crop;
mCore->mSharedBufferCache.transform = transform;
mCore->mSharedBufferCache.scalingMode = static_cast<uint32_t>(
scalingMode);
mCore->mSharedBufferCache.dataspace = dataSpace;
}
output->bufferReplaced = false;
if (mCore->mQueue.empty()) {
// When the queue is empty, we can ignore mDequeueBufferCannotBlock
// and simply queue this buffer
mCore->mQueue.push_back(item);
frameAvailableListener = mCore->mConsumerListener;
} else {
// When the queue is not empty, we need to look at the last buffer
// in the queue to see if we need to replace it
const BufferItem& last = mCore->mQueue.itemAt(
mCore->mQueue.size() - 1);
if (last.mIsDroppable) {
if (!last.mIsStale) {
mSlots[last.mSlot].mBufferState.freeQueued();
// After leaving shared buffer mode, the shared buffer will
// still be around. Mark it as no longer shared if this
// operation causes it to be free.
if (!mCore->mSharedBufferMode &&
mSlots[last.mSlot].mBufferState.isFree()) {
mSlots[last.mSlot].mBufferState.mShared = false;
}
// Don't put the shared buffer on the free list.
if (!mSlots[last.mSlot].mBufferState.isShared()) {
mCore->mActiveBuffers.erase(last.mSlot);
mCore->mFreeBuffers.push_back(last.mSlot);
output->bufferReplaced = true;
}
}
// Make sure to merge the damage rect from the frame we're about
// to drop into the new frame's damage rect.
if (last.mSurfaceDamage.bounds() == Rect::INVALID_RECT ||
item.mSurfaceDamage.bounds() == Rect::INVALID_RECT) {
item.mSurfaceDamage = Region::INVALID_REGION;
} else {
item.mSurfaceDamage |= last.mSurfaceDamage;
}
// Overwrite the droppable buffer with the incoming one
mCore->mQueue.editItemAt(mCore->mQueue.size() - 1) = item;
frameReplacedListener = mCore->mConsumerListener;
} else {
mCore->mQueue.push_back(item);
frameAvailableListener = mCore->mConsumerListener;
}
}
......
}
这个函数也比较多,同样分部分来看,首先看第一部分:
Surface
传递过来的input
里面封装的buffer信息。scalingMode
,缩放模式,当buffer内容和屏幕不成比例时采用什么方式处理,scalingMode
可以通过函数native_window_set_scaling_mode
进行设置,如果没有设置直接返回BAD_VALUE
。BufferQueue
是否被弃用,buffer是否连接到BufferQueue
,BufferSlot
下标是否合法,当前这个BufferSlot
状态是否是DEQUEUE
,mRequestBufferCalled
是否为true(即此BufferSlot
是否调用了requestBuffer
函数)。graphicBuffer
(此次queue的具体buffer),bufferRect
(根据当前graphicBuffer
创建的矩形区域),croppedRect
(裁剪区域,值为crop和bufferRect相交部分,其实这里的crop.intersect(bufferRect, &croppedRect)
和前面Surface里面mCrop.intersect(Rect(buffer->width, buffer->height), &crop)
会得到同样的裁剪区域,只要裁剪的部分包含在当前buffer之内)。BufferSlot
状态修改为QUEUE
,并修改一些状态值。BufferSlot
以及GraphicBuffer
的信息进一步封装到BufferItem
,而BufferItem
又存储在BufferQueueCore
的mQueue
中,所以一般我们说的生产者-消费者模型中的buffer队列,更具体的话就是mQueue
。如果mQueue
为空就直接将BufferItem
添加进去,并将BufferQueueCore
的mConsumerListener
给到frameAvailableListener
,从AndroidQ 图形系统(2)生产者-消费者模型知道,mConsumerListener
是在ConsumerBase
的构造函数中传递的BufferQueue::ProxyConsumerListener
。mQueue
非空的情况,首先取到mQueue
尾部的BufferItem
,根据mIsDroppable
值查看是否需要丢弃,什么情况下需要丢弃?有很多种情况,常见就是同步模式即mAsyncMode
为true,至于需要丢弃的情况怎么处理的我这里不去看了,对于不需要丢弃的情况总是将BufferItem
放入mQueue
的尾部。接着看剩下的部分:
status_t BufferQueueProducer::queueBuffer(int slot,
const QueueBufferInput &input, QueueBufferOutput *output) {
......
mCore->mBufferHasBeenQueued = true;
mCore->mDequeueCondition.notify_all();
mCore->mLastQueuedSlot = slot;
output->width = mCore->mDefaultWidth;
output->height = mCore->mDefaultHeight;
output->transformHint = mCore->mTransformHint;
output->numPendingBuffers = static_cast<uint32_t>(mCore->mQueue.size());
output->nextFrameNumber = mCore->mFrameCounter + 1;
ATRACE_INT(mCore->mConsumerName.string(),
static_cast<int32_t>(mCore->mQueue.size()));
mCore->mOccupancyTracker.registerOccupancyChange(mCore->mQueue.size());
// Take a ticket for the callback functions
callbackTicket = mNextCallbackTicket++;
VALIDATE_CONSISTENCY();
} // Autolock scope
// It is okay not to clear the GraphicBuffer when the consumer is SurfaceFlinger because
// it is guaranteed that the BufferQueue is inside SurfaceFlinger's process and
// there will be no Binder call
if (!mConsumerIsSurfaceFlinger) {
item.mGraphicBuffer.clear();
}
// Call back without the main BufferQueue lock held, but with the callback
// lock held so we can ensure that callbacks occur in order
int connectedApi;
sp<Fence> lastQueuedFence;
{ // scope for the lock
std::unique_lock<std::mutex> lock(mCallbackMutex);
while (callbackTicket != mCurrentCallbackTicket) {
mCallbackCondition.wait(lock);
}
if (frameAvailableListener != nullptr) {
frameAvailableListener->onFrameAvailable(item);
} else if (frameReplacedListener != nullptr) {
frameReplacedListener->onFrameReplaced(item);
}
connectedApi = mCore->mConnectedApi;
lastQueuedFence = std::move(mLastQueueBufferFence);
mLastQueueBufferFence = std::move(acquireFence);
mLastQueuedCrop = item.mCrop;
mLastQueuedTransform = item.mTransform;
++mCurrentCallbackTicket;
mCallbackCondition.notify_all();
}
// Update and get FrameEventHistory.
nsecs_t postedTime = systemTime(SYSTEM_TIME_MONOTONIC);
NewFrameEventsEntry newFrameEventsEntry = {
currentFrameNumber,
postedTime,
requestedPresentTimestamp,
std::move(acquireFenceTime)
};
addAndGetFrameTimestamps(&newFrameEventsEntry,
getFrameTimestamps ? &output->frameTimestamps : nullptr);
// Wait without lock held
if (connectedApi == NATIVE_WINDOW_API_EGL) {
// Waiting here allows for two full buffers to be queued but not a
// third. In the event that frames take varying time, this makes a
// small trade-off in favor of latency rather than throughput.
lastQueuedFence->waitForever("Throttling EGL Production");
}
return NO_ERROR;
}
这部分主要做了如下事情:
mBufferHasBeenQueued
一个状态值,代表buffer已经queued
,mDequeueCondition
是C++条件变量用作等待/唤醒,这里调用notify_all
会唤醒调用了wait
的线程,即在dequeueBuffer
函数中因为tooManyBuffers
而陷入等待状态的线程,当一块buffer被queued
之后就可以继续dequeue
了,output
最终会返回给Surface
进程,mConsumerIsSurfaceFlinger
表示消费者是否SurfaceFlinger
进程,默认值为true。frameAvailableListener
不为空则调用onFrameAvailable
回调函数通知有buffer可以进行消费,这个函数会经过一系列调用,最终的实现类是BufferQueueLayer
(一种最常用的Layer),最后如果连接BufferQueue
的方式为NATIVE_WINDOW_API_EGL
,会通过Fence陷入等待,这里会一直等,知道上一帧绘制完成,图形系统的生产消费非常依赖Fence同步机制,Android引入三重缓冲之后,GPU,CPU,显示硬件可以各持有一个buffer工作,例如当CPU持有的buffer数据已经处理完了,但此时GUP,显示硬件还在使用buffer,CPU并不能立马申请下一个buffer,需要等待,我这里对Fence不了解,所以文中出现的和Fence同步相关的代码就略过了。到此queueBuffer
函数已经分析完毕,代码看着多,逻辑比较简单,就是将Surface
进程数据传过来,然后封装成BufferItem
,放入BufferQueueCore
的mQueue
中,再通过frameAvailableListener
通知消费者去消费。
接着回到Surface
看剩下的代码,
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
......
if (err != OK) {
ALOGE("queueBuffer: error queuing buffer to SurfaceTexture, %d", err);
}
if (mEnableFrameTimestamps) {
mFrameEventHistory->applyDelta(output.frameTimestamps);
// Update timestamps with the local acquire fence.
// The consumer doesn't send it back to prevent us from having two
// file descriptors of the same fence.
mFrameEventHistory->updateAcquireFence(mNextFrameNumber,
std::make_shared<FenceTime>(fence));
// Cache timestamps of signaled fences so we can close their file
// descriptors.
mFrameEventHistory->updateSignalTimes();
}
mLastFrameNumber = mNextFrameNumber;
mDefaultWidth = output.width;
mDefaultHeight = output.height;
mNextFrameNumber = output.nextFrameNumber;
// Ignore transform hint if sticky transform is set or transform to display inverse flag is
// set.
if (mStickyTransform == 0 && !transformToDisplayInverse()) {
mTransformHint = output.transformHint;
}
mConsumerRunningBehind = (output.numPendingBuffers >= 2);
if (!mConnectedToCpu) {
// Clear surface damage back to full-buffer
mDirtyRegion = Region::INVALID_REGION;
}
if (mSharedBufferMode && mAutoRefresh && mSharedBufferSlot == i) {
mSharedBufferHasBeenQueued = true;
}
mQueueBufferCondition.broadcast();
if (CC_UNLIKELY(atrace_is_tag_enabled(ATRACE_TAG_GRAPHICS))) {
static FenceMonitor gpuCompletionThread("GPU completion");
gpuCompletionThread.queueFence(fence);
}
return err;
这后面部分其实没什么好说的了,会更新一些变量,清空脏区域,通过条件变量mQueueBufferCondition
唤醒等待下一帧的线程,处理一下Fence。