Android4.2.2 SurfaceFlinger之图形渲染queueBuffer实现和VSYNC的存在感
本文均属自己阅读源码的点滴总结，转账请注明出处谢谢。
欢迎和大家交流。qq:1037701636 email:[email protected]
Android源码版本Version：4.2.2; 硬件平台 全志A31
前一博文总结了Android4.2.2 SurfaceFlinger之图形缓存区申请与分配dequeueBuffer的实现，由于受到OpenGL Es的中介作用（内部实现图层绘制并写入到分配好的图形缓存中去），eglSwapBuffers()函数内部的实现就是如此。好了作为生产者以及使用dequeueBuffer获取了图形缓存并写入了绘图数据，这下就该是渲染的过程queueBuffer来看看他的实现：
status_t BufferQueue::queueBuffer(int buf,
const QueueBufferInput& input, QueueBufferOutput* output) {
ATRACE_CALL();
.......
{ // scope for the lock
Mutex::Autolock lock(mMutex);
if (mAbandoned) {
ST_LOGE("queueBuffer: SurfaceTexture has been abandoned!");
return NO_INIT;
}
int maxBufferCount = getMaxBufferCountLocked();
if (buf < 0 || buf >= maxBufferCount) {
ST_LOGE("queueBuffer: slot index out of range [0, %d]: %d",
maxBufferCount, buf);
return -EINVAL;
} else if (mSlots[buf].mBufferState != BufferSlot::DEQUEUED) {//非DEQUEUD，未被客户端获取
ST_LOGE("queueBuffer: slot %d is not owned by the client "
"(state=%d)", buf, mSlots[buf].mBufferState);
return -EINVAL;
} else if (!mSlots[buf].mRequestBufferCalled) {
ST_LOGE("queueBuffer: slot %d was enqueued without requesting a "
"buffer", buf);
return -EINVAL;
}
const sp& graphicBuffer(mSlots[buf].mGraphicBuffer);
Rect bufferRect(graphicBuffer->getWidth(), graphicBuffer->getHeight());
Rect croppedCrop;
crop.intersect(bufferRect, &croppedCrop);
if (croppedCrop != crop) {
ST_LOGE("queueBuffer: crop rect is not contained within the "
"buffer in slot %d", buf);
return -EINVAL;
}
if (mSynchronousMode) {
// In synchronous mode we queue all buffers in a FIFO.
mQueue.push_back(buf);
// Synchronous mode always signals that an additional frame should
// be consumed.
listener = mConsumerListener;//获取消费者监听
} else {
// In asynchronous mode we only keep the most recent buffer.
if (mQueue.empty()) {
mQueue.push_back(buf);
// Asynchronous mode only signals that a frame should be
// consumed if no previous frame was pending. If a frame were
// pending then the consumer would have already been notified.
listener = mConsumerListener;
} else {
Fifo::iterator front(mQueue.begin());
// buffer currently queued is freed
mSlots[*front].mBufferState = BufferSlot::FREE;
// and we record the new buffer index in the queued list
*front = buf;
}
}
mSlots[buf].mTimestamp = timestamp;
mSlots[buf].mCrop = crop;
mSlots[buf].mTransform = transform;
mSlots[buf].mFence = fence;
switch (scalingMode) {
case NATIVE_WINDOW_SCALING_MODE_FREEZE:
case NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW:
case NATIVE_WINDOW_SCALING_MODE_SCALE_CROP:
break;
default:
ST_LOGE("unknown scaling mode: %d (ignoring)", scalingMode);
scalingMode = mSlots[buf].mScalingMode;
break;
}
mSlots[buf].mBufferState = BufferSlot::QUEUED;
mSlots[buf].mScalingMode = scalingMode;
mFrameCounter++;
mSlots[buf].mFrameNumber = mFrameCounter;
mBufferHasBeenQueued = true;
mDequeueCondition.broadcast();
output->inflate(mDefaultWidth, mDefaultHeight, mTransformHint,
mQueue.size());
ATRACE_INT(mConsumerName.string(), mQueue.size());
}
......
if (listener != 0) {
listener->onFrameAvailable();//发布当前帧可以给消费者
}
}
step1: 根据bufferSlot[]的索引值找到对应的buffer后，对其做是否处于DEQUEUED状态，因为只有被客户端获取后才能去进一步的渲染。
step2: sp
BufferQueue中的一个成员变量mConsumerListener是queueBuffer的主导，这个变量在哪里何时创建的呢，来看这里：
ConsumerBase::ConsumerBase(const sp& bufferQueue) :
mAbandoned(false),
mBufferQueue(bufferQueue) {
// Choose a name using the PID and a process-unique ID.
mName = String8::format("unnamed-%d-%d", getpid(), createProcessUniqueId());
// Note that we can't create an sp<...>(this) in a ctor that will not keep a
// reference once the ctor ends, as that would cause the refcount of 'this'
// dropping to 0 at the end of the ctor. Since all we need is a wp<...>
// that's what we create.
wp listener;
sp proxy;
listener = static_cast(this);
proxy = new BufferQueue::ProxyConsumerListener(listener);//新建一个监听代理
status_t err = mBufferQueue->consumerConnect(proxy);//创建一个ConsumerListener代理
if (err != NO_ERROR) {
CB_LOGE("SurfaceTexture: error connecting to BufferQueue: %s (%d)",
strerror(-err), err);
} else {
mBufferQueue->setConsumerName(mName);
}
}
在ConsumerBase的构造函数之中，而该类被SurfaceTexture继承，且SurfaceFlinger在新建Layer时创建new SurfaceTexture时进行的。这里可以看到调用了一个consumerConnect函数：
status_t BufferQueue::consumerConnect(const sp& consumerListener) {
.......
mConsumerListener = consumerListener;
return OK;
} 这里就看到了BufferQueue的成员变量mConsumerListener完成了赋值初始化，从调用可知，传入的proxy为BufferQueue的一个内部类ProxyConsumerListener，理解为消费者监听代理。
而实际上的这个proxy自己也是有一个成员变量mConsumerListener即为上面传入的listener（这里的this是什么？，当初ConsumerBase是基于new SurfaceTexture()来构造 ，该类继承了ConsumerBase，故可以理解为这个this实际是SurfaceFLinger侧的SurfaceTexture对象)，为BufferQueue的内部类ConsumerListener。
BufferQueue::ProxyConsumerListener::ProxyConsumerListener(
const wp& consumerListener):
mConsumerListener(consumerListener) {}
step3:回到queueBuffer函数的最后listener->onFrameAvailable()
由于listener是ConsumerListener类，proxy是ProxyConsumerListener，且继承public与ConsumerListener，故通过上面的分析最终调用的是proxy->onFrameAvailable，最终来到这里：
void BufferQueue::ProxyConsumerListener::onFrameAvailable() {
sp listener(mConsumerListener.promote());
if (listener != NULL) {
listener->onFrameAvailable();//调用消费者consumerbase的onFrameAvailable进行调用
}
} 这里的mConsumerListerner成员对象其实是指向派生类对象ConsumerBase的基类指针，故最终回到了ConsumerBase的onFrameAvailable().
因此上面的理解应该是一个proxy监听代理，最终还是提交给消费者来处理，而最下层的消费者就是SurfaceTexture。
step4:由于SurfaceTexture没有重载，则调用的ConsumerBase如下：
void ConsumerBase::onFrameAvailable() {
CB_LOGV("onFrameAvailable");
sp listener;
{ // scope for the lock
Mutex::Autolock lock(mMutex);
listener = mFrameAvailableListener;
}
if (listener != NULL) {
CB_LOGV("actually calling onFrameAvailable");
listener->onFrameAvailable();
}
}
这里有出现了一个FrameAvailableListener类，帧可用监听类，那这个ConsumerBase的成员变量mFrameAvailableListener是在哪里初始化的呢，回到这里？
void Layer::onFirstRef() { LayerBaseClient::onFirstRef();//基类LayerBaseClient
struct FrameQueuedListener : public SurfaceTexture::FrameAvailableListener {//内部类继承FrameAvailableListener FrameQueuedListener(Layer* layer) : mLayer(layer) { } private: wp
// Creates a custom BufferQueue for SurfaceTexture to use sp
mSurfaceTexture->setConsumerUsageBits(getEffectiveUsage(0)); mSurfaceTexture->setFrameAvailableListener(new FrameQueuedListener(this));//新建立一个帧队列监听 mSurfaceTexture->setSynchronousMode(true);//支持同步模式
#ifdef TARGET_DISABLE_TRIPLE_BUFFERING #warning "disabling triple buffering" mSurfaceTexture->setDefaultMaxBufferCount(2); #else mSurfaceTexture->setDefaultMaxBufferCount(3); #endif
const sp
这里出现了一个和SurfaceTexture相关的帧可用监听设置函数setFrameAvailableListener：
void ConsumerBase::setFrameAvailableListener(
const sp& listener) {
CB_LOGV("setFrameAvailableListener");
Mutex::Autolock lock(mMutex);
mFrameAvailableListener = listener;
这里可以看到listener = new FrameQueuedListener()对象，故最终调用的是FrameQueuedListener->onFrameAvailable()函数，而该类其实是Layer::onFirstRef的内部类FrameQueuedListener，该函数最终还是调用如下：
virtual void onFrameAvailable() {
sp that(mLayer.promote());
if (that != 0) {
that->onFrameQueued();//调用Layer的onFrameQueued
}
} 而这个mLayer是在对象FrameQueuedListener传入的this参数，这个this就是传入的layer对象，故最终这个that->onFrameQueued就回到了Layer::onFrameQueued函数。
step5:回到了熟悉的Layer处的调用
void Layer::onFrameQueued() {
android_atomic_inc(&mQueuedFrames);
mFlinger->signalLayerUpdate();//发送给SF Layer图层更新
}这里的结果很显然，一切还得有SurfaceFlinger来完成图形缓存区的渲染，发出图层更新的信号：
void SurfaceFlinger::signalLayerUpdate() {
mEventQueue.invalidate();
}
mEventQueue又涉及到了SurfaceFLinger的消息处理机制，意味着又要触发一个Event的发生：
void MessageQueue::invalidate() {
#if INVALIDATE_ON_VSYNC
mEvents->requestNextVsync();
#else
mHandler->dispatchInvalidate();
#endif
}这里执行requestNextVsync()函数，即请求下一个VSYNC。那么这个mEvents是什么呢？可以在这里看到被初始化
void MessageQueue::setEventThread(const sp& eventThread)
{
mEventThread = eventThread;
mEvents = eventThread->createEventConnection();//建立连接
....} sp EventThread::createEventConnection() const {
return new Connection(const_cast(this));
} 这个函数内部新建了一个Connection类对象，该类是EventThread的内部类继承了BnDisplayEventConnection类。这里有必要深入的看看connection的构造过程：
EventThread::Connection::Connection(
const sp& eventThread)
: count(-1), mEventThread(eventThread), mChannel(new BitTube())
{
}
该类继承RefBase则执行onFirstRef()函数，看看他做了什么特别的事情：
void EventThread::Connection::onFirstRef() {
// NOTE: mEventThread doesn't hold a strong reference on us
mEventThread->registerDisplayEventConnection(this);//将随着eventthread新建的connection对象注册到mDisplayEventConnections
}将这个新建出来的Connection对象this注册到显示事件中去：
status_t EventThread::registerDisplayEventConnection(
const sp& connection) {
Mutex::Autolock _l(mLock);
mDisplayEventConnections.add(connection);
mCondition.broadcast();
return NO_ERROR;
}
最终是将这个connection维护在了一个SortedVector< wp
上述分析后，可知最终requestNextVsync调用的是Connection的requestNextVsync来完成的：
void EventThread::Connection::requestNextVsync() {
mEventThread->requestNextVsync(this);
}
mEventThread是直接新建时传入的this，且为EventThread；故最终调用的是EventThread类的成员函数requestNextVsync()
step6:EventThread的requestNextVsync函数
void EventThread::requestNextVsync(
const sp& connection) {
Mutex::Autolock _l(mLock);
if (connection->count < 0) {
connection->count = 0;
mCondition.broadcast();//条件满足时，发出一个广播请求下一个VSYNC
}
}
这里是将count清0，并触发一个进程锁的解锁，这个锁等待在EventThread的threadLoop()中，以EventThread::waitForEvent()函数的形式睡眠，下面来看部分代码：
Vector< sp > EventThread::waitForEvent(
DisplayEventReceiver::Event* event){
// find out connections waiting for events
size_t count = mDisplayEventConnections.size();
for (size_t i=0 ; i connection(mDisplayEventConnections[i].promote());
if (connection != NULL) {
bool added = false;
if (connection->count >= 0) {
// we need vsync events because at least
// one connection is waiting for it
waitForVSync = true;
if (timestamp) {
// we consume the event only if it's time
// (ie: we received a vsync event)
if (connection->count == 0) {
// fired this time around
connection->count = -1;
signalConnections.add(connection);
added = true;
} else if (connection->count == 1 ||
(vsyncCount % connection->count) == 0) {
// continuous event, and time to report it
signalConnections.add(connection);
added = true;
}
}
}
while (signalConnections.isEmpty())............
}
在获得之前注册到mDisplayEventConnections数组之中connection对象，依据之前的requestNextVsync()，他会将count=0，故这里执行signalConnections.add(connection);
有了连击信号后，直接退出waitEvent()。
step7:到这里我们需要回到Android4.2.2 SurfaceFlinger的相关事件和消息处理机制这里，因为我们知道如果要渲染当前的图像，需要接受到底层硬件返回的一个VSYNC，而这里有HWComposer来完成（当然硬件不支持时会由一个VSyncThread来软件模拟）。
在SF运行时就会创建一个HWComposer：
mHwc = new HWComposer(this,
*static_cast(this));//新建一个软硬件合成器HWComposer
在HWComposer的构造函数内，可以看到向hwcomposer的HAL层注册了回调业务函数：
if (mHwc) {//如果支持硬件hw
ALOGI("Using %s version %u.%u", HWC_HARDWARE_COMPOSER,
(hwcApiVersion(mHwc) >> 24) & 0xff,
(hwcApiVersion(mHwc) >> 16) & 0xff);//1.1版本
if (mHwc->registerProcs) {
mCBContext->hwc = this;
mCBContext->procs.invalidate = &hook_invalidate;
mCBContext->procs.vsync = &hook_vsync;//相关hwc的回调函数初始化，用于向上层发出VSYNC信号
if (hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1))
mCBContext->procs.hotplug = &hook_hotplug;
else
mCBContext->procs.hotplug = NULL;
memset(mCBContext->procs.zero, 0, sizeof(mCBContext->procs.zero));
mHwc->registerProcs(mHwc, &mCBContext->procs);//向HAL注册回调函数
}很容易的可以看到底层硬件产生一个硬件的VSYNC时，传入的回调函数即为hook_vsync。那么底层硬件又是如何反馈回这个VSYNC信号的呢？
step8: VSYNC的硬件信号捕获进程分析
在之前已经说到registerProcs会将回调函数进行注册，而这个不单单是注册这么简单，我们来看看：
static void hwc_registerProcs(struct hwc_composer_device_1* dev,
hwc_procs_t const* procs)
{
ALOGI("%s", __FUNCTION__);
hwc_context_t* ctx = (hwc_context_t*)(dev);
if(!ctx) {
ALOGE("%s: Invalid context", __FUNCTION__);
return;
}
ctx->proc = procs;
// Now that we have the functions needed, kick off
// the uevent & vsync threads
init_uevent_thread(ctx);
init_vsync_thread(ctx);
}果然这里出现了线程的存在感，即这里新建了两个线程而init_vsync_thread就是轮询捕获VSYNC的真正线程：
void init_vsync_thread(hwc_context_t* ctx)
{
int ret;
pthread_t vsync_thread;
ALOGI("Initializing VSYNC Thread");
ret = pthread_create(&vsync_thread, NULL, vsync_loop, (void*) ctx);
if (ret) {
ALOGE("%s: failed to create %s: %s", __FUNCTION__,
HWC_VSYNC_THREAD_NAME, strerror(ret));
}
}这里创建了一个vsync_loop的线程函数，且传入的参数为回调函数的内容。进入线程后，基本就是和内核进行交互，捕获VSYNC，而且肯定是处于死循环中。
static void *vsync_loop(void *param)
{
const char* vsync_timestamp_fb0 = "/sys/class/graphics/fb0/vsync_event";
const char* vsync_timestamp_fb1 = "/sys/class/graphics/fb1/vsync_event";
int dpy = HWC_DISPLAY_PRIMARY;
.....
do
{
ctx->proc->vsync(ctx->proc, dpy, cur_timestamp);
}while(true);
}
}从而属于SurfaceFlinger进程 一个VSYNC线程就在实时监听VSYNC的存在，并回调给vsync()函数，即注册的hook_sync().
step9: hook_sync()的作用
void HWComposer::hook_vsync(const struct hwc_procs* procs, int disp,
int64_t timestamp) {
cb_context* ctx = reinterpret_cast(
const_cast(procs));
ctx->hwc->vsync(disp, timestamp);
}
继续该函数的执行，回到了HWComposer中来执行
void HWComposer::vsync(int disp, int64_t timestamp) {
ATRACE_INT("VSYNC", ++mVSyncCount&1);
mEventHandler.onVSyncReceived(disp, timestamp);//调用SF的onVSyncReceived
Mutex::Autolock _l(mLock);
mLastHwVSync = timestamp;
}而这里利用的mEventHandle是对象就是SurfaceFlinger对象，故转到SurfaceFlinger::onVSyncReceived函数来执行
void SurfaceFlinger::onVSyncReceived(int type, nsecs_t timestamp) {
if (mEventThread == NULL) {
// This is a temporary workaround for b/7145521. A non-null pointer
// does not mean EventThread has finished initializing, so this
// is not a correct fix.
ALOGW("WARNING: EventThread not started, ignoring vsync");
return;
}
if (uint32_t(type) < DisplayDevice::NUM_DISPLAY_TYPES) {
// we should only receive DisplayDevice::DisplayType from the vsync callback
mEventThread->onVSyncReceived(type, timestamp);//发出一个VSYNC接收
}
}
上面函数利用SF处的事件处理线程进一步执行，完成了对当前的VSYNC事件进行初始化，并提交一个broadcast，供相应的阻塞线程进行唤醒。
void EventThread::onVSyncReceived(int type, nsecs_t timestamp) {
ALOGE_IF(type >= HWC_DISPLAY_TYPES_SUPPORTED,
"received event for an invalid display (id=%d)", type);
Mutex::Autolock _l(mLock);
if (type < HWC_DISPLAY_TYPES_SUPPORTED) {
mVSyncEvent[type].header.type = DisplayEventReceiver::DISPLAY_EVENT_VSYNC;
mVSyncEvent[type].header.id = type;
mVSyncEvent[type].header.timestamp = timestamp;
mVSyncEvent[type].vsync.count++;
mCondition.broadcast();
}
}唤醒的线程就是EventThread的waitEvent()函数，故回到step6处继续分析。
step10:VSYNC的事件处理
VSYNC的处理过程在Android4.2.2 SurfaceFlinger的相关事件和消息处理机制已经详细的分析过，这里不做过多的分析。要补充的是只有一个connection对象存在时，才会提交一个Event事件：
bool EventThread::threadLoop() {
DisplayEventReceiver::Event event;
Vector< sp > signalConnections;
signalConnections = waitForEvent(&event);//轮询等待event的发生，一般是硬件的请求
const size_t count = signalConnections.size();
for (size_t i=0 ; i& conn(signalConnections[i]);
// now see if we still need to report this event
status_t err = conn->postEvent(event);//发送事件，一般的硬件触发了事件的发生 很容易看到只有需要渲染的一个buffer发出connection时即让waitEvent从睡眠中醒过来mCondition.wait(mLock);
简单的几个分支情况如下：
1.如果没有VSYNC时，唤醒waitEvent()中的阻塞函数，且会执行psotEvent()，但是没有Event的相关信息。
2.如果只有VSYNC，则signalConnections不会被add相关的connection，只会继续睡眠，唤醒，睡眠。
3.从而只有在step6中那样，唤醒线程且添加对象connection到signalConnections里面，再次进入睡眠，等着VSYNC唤醒后直接退出循环，最终才会提交一次Event
step11: 最终事件被eventReceiver()回调后并发出一个消息，消息处理由下面的函数来处理
int MessageQueue::eventReceiver(int fd, int events) {
ssize_t n;
DisplayEventReceiver::Event buffer[8];
while ((n = DisplayEventReceiver::getEvents(mEventTube, buffer, 8)) > 0) {//得到事件的数据
for (int i=0 ; idispatchInvalidate();
#else
mHandler->dispatchRefresh();//刷新
#endif
break;
}
}
}
return 1;
} void MessageQueue::Handler::dispatchInvalidate() {
if ((android_atomic_or(eventMaskInvalidate, &mEventMask) & eventMaskInvalidate) == 0) {
mQueue.mLooper->sendMessage(this, Message(MessageQueue::INVALIDATE));
}
}发出的消息类型为INVALIDATE。
void MessageQueue::Handler::handleMessage(const Message& message) {//事件消息处理机制
switch (message.what) {
case INVALIDATE:
android_atomic_and(~eventMaskInvalidate, &mEventMask);
mQueue.mFlinger->onMessageReceived(message.what);
break;
case REFRESH:
android_atomic_and(~eventMaskRefresh, &mEventMask);
mQueue.mFlinger->onMessageReceived(message.what);//线程接收到了消息后处理
break;
}
}
首先是执行INVALIDATE函数，先来看看他完成的任务，最终发现他还是提交给SurfaceFlinger来完成：
void SurfaceFlinger::onMessageReceived(int32_t what) {//收到消息
ATRACE_CALL();
switch (what) {
case MessageQueue::INVALIDATE://SurfaceFlinger的处理
handleMessageTransaction();
handleMessageInvalidate();
signalRefresh();
break;
case MessageQueue::REFRESH:
handleMessageRefresh();
break;
}
}
上面的函数分别是对当前的绘图的信息发生变化时来做处理的，比如z轴发生变化，透传变化等等。
void SurfaceFlinger::handleMessageTransaction() {
uint32_t transactionFlags = peekTransactionFlags(eTransactionMask);
if (transactionFlags) {
handleTransaction(transactionFlags);//事务处理
}
}
void SurfaceFlinger::handleMessageInvalidate() {
ATRACE_CALL();
handlePageFlip();//
}
处理完后最终发出signalRefresh();信号来完成最终的绘图。
void SurfaceFlinger::signalRefresh() {
mEventQueue.refresh();//刷新
}void MessageQueue::refresh() {
#if INVALIDATE_ON_VSYNC
mHandler->dispatchRefresh();
#else
mEvents->requestNextVsync();
#endif
}最终这里发出REFRESH类型的消息
void MessageQueue::Handler::dispatchRefresh() {
if ((android_atomic_or(eventMaskRefresh, &mEventMask) & eventMaskRefresh) == 0) {
mQueue.mLooper->sendMessage(this, Message(MessageQueue::REFRESH));
}
}而最终的处理同理还是由SurfaceFlinger::onMessageReceived(int32_t what)里的handleMessageRefresh()来完成处理：
void SurfaceFlinger::handleMessageRefresh() {//处理layer的刷新，实际是调用SF绘图
ATRACE_CALL();
preComposition();
rebuildLayerStacks();
setUpHWComposer();
doDebugFlashRegions();
doComposition();
postComposition();//写入到FrameBuffer中
}
最终依旧上述函数完成绘图送显。
到这里就完成了queueBuffer的整个流程，细节上的东西比较多，讲的也比较粗。
补充：
SurfaceFlinger本地的窗口和应用侧的窗口的存在性，都依赖于SurfaceTextureClient类关联在一起。
