SurfaceFlinger是android平台的显示服务,为移动互联网时代的内容呈现和交互提供了平台级的基础。本文以Android4.2的源代码和架构为例,详细介绍SurfaceFlinger服务。
SurfaceFlinger服务的源代码位于frameworks/native/cmds/surfaceflinger下:
int main(int argc, char** argv) { SurfaceFlinger::publishAndJoinThreadPool(true); // When SF is launched in its own process, limit the number of // binder threads to 4. ProcessState::self()->setThreadPoolMaxThreadCount(4); return 0; }
为android平台上最常见的native BinderService启动方式。
下边结合binder上层接口粗略分析该段程序。
将"SurfaceFlinger::publishAndJoinThreadPool(true);"
扩展开变为:
sp<IServiceManager> sm(defaultServiceManager()); sm->addService(String16(SERVICE::getServiceName()), new SERVICE(), allowIsolated); // 向servicemanager注册SurfaceFinger服务,以“SurfaceFlinger”作为名字/标识 ProcessState::self()->startThreadPool(); IPCThreadState::self()->joinThreadPool();
ProcessState为每进程的记录binder进程状态的类。主要职责在于:
ProcessState::self()->startThreadPool();
IPCThreadState::self()->joinThreadPool();
Parcel mIn; Parcel mOut;
这样子主线程也进入looper模式,现在进入joinThreadPool方法:
binder通信的主要细节就在此方法中,主要思想便是通过talkWithDriver来取到本进程接收到的binder请求并将其反序列化为mIn身上,读出cmd并执行之,摘取几个重要的cmd如下:
BR_TRANSACTION:代表一次binder事务,最常见的便是RPC调用了
binder_transaction_data tr; ...... if (tr.target.ptr) { sp<BBinder> b((BBinder*)tr.cookie); const status_t error = b->transact(tr.code, buffer, &reply, tr.flags); if (error < NO_ERROR) reply.setError(error); } else { const status_t error = the_context_object->transact(tr.code, buffer, &reply, tr.flags); if (error < NO_ERROR) reply.setError(error); }
binder_transaction_data身上带有服务端的BBinder(client端第一次从servicemanager查询到服务端时被告知并由binder driver记录),
当tr.target.ptr非空时,通过该BBinder对象的transact函数来完成RPC。否则意味着对context_object(通常为本进程内的BpServiceManager对象,且对应handle为0 )的请求。
BBinder的子类通过onTransact函数来服务各种RPC调用。在本文中,便是SurfaceFlinger对象的onTransact函数来应对client的请求。
BR_SPAWN_LOOPER:
前边启动过程中,binder thread包括主线程(SurfaceFlinger对象构造过程中也创建了额外辅助进程,但不是binder thread,排除在外,在本文后半部分分析)一共是两个。当有多个client请求时,势必binder thread不够用(binder driver簿记binder thread的各种信息),此时binder driver便post BR_SPAWN_LOOPER类型的cmd,让本进程预先孵化下一个binder thread,且此后孵化的线程不会进入looper模式(主要处理线程),只是登记为looper(目前在driver中未看出来两者明显区别)
另外,当本进程需要其他服务的协助的时候呢,本进程也需要作为client来发起binder请求,此时便用到了BpBinder::transact()函数,BpBinder成员mHandle用来引用远端对象,作为参数传递给IPCThreadState::self()->transact()来完成。其原理与前述接受binder请求的过程相反,不再赘述。
此步骤是前述启动步骤中的一小步,但也是展现出特定service与众不同的最大一步。
如文件第一部分所列类图所示,SurfaceFlinger继承自多个基类:
class SurfaceFlinger : public BinderService<SurfaceFlinger>, public BnSurfaceComposer, private IBinder::DeathRecipient, private Thread, private HWComposer::EventHandler
在ServiceManager addservice时第一次引用刚创建的SurfaceFinger对象,其基类RefBase的onFirstRef被调用:
void SurfaceFlinger::onFirstRef() { mEventQueue.init(this); run("SurfaceFlinger", PRIORITY_URGENT_DISPLAY); // Wait for the main thread to be done with its initialization mReadyToRunBarrier.wait(); }
接下来启动自己(SurfaceFlinger也是Thread对象),
然后主线程阻塞住直到ReadyToRun(Thread真正启动前的一次性初始化函数)被调用。
接下来咱们转到ReadyToRun函数,鉴于该函数如一枚知性的熟女——深有内涵:
ALOGI( "SurfaceFlinger's main thread ready to run. " "Initializing graphics H/W..."); // initialize EGL for the default display mEGLDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY); eglInitialize(mEGLDisplay, NULL, NULL); // Initialize the H/W composer object. There may or may not be an // actual hardware composer underneath. mHwc = new HWComposer(this, *static_cast<HWComposer::EventHandler *>(this)); // initialize the config and context EGLint format = mHwc->getVisualID(); mEGLConfig = selectEGLConfig(mEGLDisplay, format); mEGLContext = createGLContext(mEGLDisplay, mEGLConfig); LOG_ALWAYS_FATAL_IF(mEGLContext == EGL_NO_CONTEXT, "couldn't create EGLContext"); // initialize our non-virtual displays for (size_t i=0 ; i<DisplayDevice::NUM_DISPLAY_TYPES ; i++) { DisplayDevice::DisplayType type((DisplayDevice::DisplayType)i); mDefaultDisplays[i] = new BBinder(); wp<IBinder> token = mDefaultDisplays[i]; // set-up the displays that are already connected if (mHwc->isConnected(i) || type==DisplayDevice::DISPLAY_PRIMARY) { // All non-virtual displays are currently considered secure. bool isSecure = true; mCurrentState.displays.add(token, DisplayDeviceState(type)); sp<FramebufferSurface> fbs = new FramebufferSurface(*mHwc, i); sp<SurfaceTextureClient> stc = new SurfaceTextureClient( static_cast< sp<ISurfaceTexture> >(fbs->getBufferQueue())); sp<DisplayDevice> hw = new DisplayDevice(this, type, isSecure, token, stc, fbs, mEGLConfig); if (i > DisplayDevice::DISPLAY_PRIMARY) { // FIXME: currently we don't get blank/unblank requests // for displays other than the main display, so we always // assume a connected display is unblanked. ALOGD("marking display %d as acquired/unblanked", i); hw->acquireScreen(); } mDisplays.add(token, hw); } } // we need a GL context current in a few places, when initializing // OpenGL ES (see below), or creating a layer, // or when a texture is (asynchronously) destroyed, and for that // we need a valid surface, so it's convenient to use the main display // for that. sp<const DisplayDevice> hw(getDefaultDisplayDevice()); // initialize OpenGL ES DisplayDevice::makeCurrent(mEGLDisplay, hw, mEGLContext); initializeGL(mEGLDisplay); // start the EventThread mEventThread = new EventThread(this); mEventQueue.setEventThread(mEventThread); // initialize our drawing state mDrawingState = mCurrentState; // We're now ready to accept clients... mReadyToRunBarrier.open(); // set initial conditions (e.g. unblank default device) initializeDisplays(); // start boot animation startBootAnim(); return NO_ERROR;
我们将其曼妙身段划分为如下:
下边摘取其中一些关键步骤分析:
EGL初始化主要通过eglGetDisplay()来实现,其参数为EGL_DEFAULT_DISPLAY(一个handle值,意义依赖于具体平台):
EGLDisplay eglGetDisplay(EGLNativeDisplayType display) { clearError(); uint32_t index = uint32_t(display); if (index >= NUM_DISPLAYS) { return setError(EGL_BAD_PARAMETER, EGL_NO_DISPLAY); } if (egl_init_drivers() == EGL_FALSE) { return setError(EGL_BAD_PARAMETER, EGL_NO_DISPLAY); } EGLDisplay dpy = egl_display_t::getFromNativeDisplay(display); return dpy; }
egl_connection_t gEGLImpl; gl_hooks_t gHooks[2];
上(gEGLImpl成员含有对gHooks的指针)。
接下来便是拿到EGLDisplay的过程,包含在egl_display_t::getFromNativeDisplay(display)中,其实现是调用上一步中装载的EGL库中的"eglGetDisplay"函数来获得真正实现并设定给egl_display_t::sDisplay[NUM_DISPLAYS].dpy,返回egl_display_t::sDisplay数组中第一个invalid项的索引作为EGLDisplay handle
接下来便是EGL初始化的过程:
EGLBoolean eglInitialize(EGLDisplay dpy, EGLint *major, EGLint *minor) { clearError(); egl_display_ptr dp = get_display(dpy); if (!dp) return setError(EGL_BAD_DISPLAY, EGL_FALSE); EGLBoolean res = dp->initialize(major, minor); return res; }其中关键步骤体现在dp->initialize中,展开它的实现:
EGLBoolean egl_display_t::initialize(EGLint *major, EGLint *minor) { Mutex::Autolock _l(lock); if (refs > 0) { if (major != NULL) *major = VERSION_MAJOR; if (minor != NULL) *minor = VERSION_MINOR; refs++; return EGL_TRUE; } #if EGL_TRACE // Called both at early_init time and at this time. (Early_init is pre-zygote, so // the information from that call may be stale.) initEglTraceLevel(); initEglDebugLevel(); #endif setGLHooksThreadSpecific(&gHooksNoContext); // 将前边从egl库中取出来的glhooks绑定到thread的TLS上,Note:bionic c为OpenGL预留了专用的TLS,see:bionic_tls.h // initialize each EGL and // build our own extension string first, based on the extension we know // and the extension supported by our client implementation egl_connection_t* const cnx = &gEGLImpl; cnx->major = -1; cnx->minor = -1; if (cnx->dso) { #if defined(ADRENO130) #warning "Adreno-130 eglInitialize() workaround" /* * The ADRENO 130 driver returns a different EGLDisplay each time * eglGetDisplay() is called, but also makes the EGLDisplay invalid * after eglTerminate() has been called, so that eglInitialize() * cannot be called again. Therefore, we need to make sure to call * eglGetDisplay() before calling eglInitialize(); */ if (i == IMPL_HARDWARE) { disp[i].dpy = cnx->egl.eglGetDisplay(EGL_DEFAULT_DISPLAY); } #endif EGLDisplay idpy = disp.dpy; if (cnx->egl.eglInitialize(idpy, &cnx->major, &cnx->minor)) { // 调用egl库中的eglInitialize //ALOGD("initialized dpy=%p, ver=%d.%d, cnx=%p", // idpy, cnx->major, cnx->minor, cnx); // display is now initialized disp.state = egl_display_t::INITIALIZED; // get the query-strings for this display for each implementation disp.queryString.vendor = cnx->egl.eglQueryString(idpy, // 查询一些meta信息 EGL_VENDOR); disp.queryString.version = cnx->egl.eglQueryString(idpy, EGL_VERSION); disp.queryString.extensions = cnx->egl.eglQueryString(idpy, EGL_EXTENSIONS); disp.queryString.clientApi = cnx->egl.eglQueryString(idpy, EGL_CLIENT_APIS); } else { ALOGW("eglInitialize(%p) failed (%s)", idpy, egl_tls_t::egl_strerror(cnx->egl.eglGetError())); } } // the query strings are per-display mVendorString.setTo(sVendorString); mVersionString.setTo(sVersionString); mClientApiString.setTo(sClientApiString); // we only add extensions that exist in the implementation char const* start = sExtensionString; // 装配扩展信息 char const* end; do { // find the space separating this extension for the next one end = strchr(start, ' '); if (end) { // length of the extension string const size_t len = end - start; if (len) { // NOTE: we could avoid the copy if we had strnstr. const String8 ext(start, len); // now look for this extension if (disp.queryString.extensions) { // if we find it, add this extension string to our list // (and don't forget the space) const char* match = strstr(disp.queryString.extensions, ext.string()); if (match && (match[len] == ' ' || match[len] == 0)) { mExtensionString.append(start, len+1); } } } // process the next extension string, and skip the space. start = end + 1; } } while (end); egl_cache_t::get()->initialize(this); // 初始化egl_cache char value[PROPERTY_VALUE_MAX]; property_get("debug.egl.finish", value, "0"); if (atoi(value)) { finishOnSwap = true; } property_get("debug.egl.traceGpuCompletion", value, "0"); if (atoi(value)) { traceGpuCompletion = true; } refs++; if (major != NULL) *major = VERSION_MAJOR; if (minor != NULL) *minor = VERSION_MINOR; mHibernation.setDisplayValid(true); // Hibernation相关 return EGL_TRUE; }
Hardware Composer的构造函数如下:
HWComposer::HWComposer( const sp<SurfaceFlinger>& flinger, EventHandler& handler) : mFlinger(flinger), mFbDev(0), mHwc(0), mNumDisplays(1), mCBContext(new cb_context), mEventHandler(handler), mVSyncCount(0), mDebugForceFakeVSync(false) { for (size_t i =0 ; i<MAX_DISPLAYS ; i++) { mLists[i] = 0; } char value[PROPERTY_VALUE_MAX]; property_get("debug.sf.no_hw_vsync", value, "0"); mDebugForceFakeVSync = atoi(value); bool needVSyncThread = true; // Note: some devices may insist that the FB HAL be opened before HWC. loadFbHalModule(); // load gralloc HAL模块 loadHwcModule(); // load hwcomposer HAL模块 if (mFbDev && mHwc && hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1)) { // close FB HAL if we don't needed it. // FIXME: this is temporary until we're not forced to open FB HAL // before HWC. framebuffer_close(mFbDev); mFbDev = NULL; } // If we have no HWC, or a pre-1.1 HWC, an FB dev is mandatory. if ((!mHwc || !hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1)) && !mFbDev) { ALOGE("ERROR: failed to open framebuffer, aborting"); abort(); } // these display IDs are always reserved for (size_t i=0 ; i<HWC_NUM_DISPLAY_TYPES ; i++) { mAllocatedDisplayIDs.markBit(i); } if (mHwc) { ALOGI("Using %s version %u.%u", HWC_HARDWARE_COMPOSER, (hwcApiVersion(mHwc) >> 24) & 0xff, (hwcApiVersion(mHwc) >> 16) & 0xff); if (mHwc->registerProcs) { // 有hardware composer的情形下,需要注册一些callback函数给它,来接受通知 mCBContext->hwc = this; mCBContext->procs.invalidate = &hook_invalidate; // invalidate hook mCBContext->procs.vsync = &hook_vsync; // vsync hook if (hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1)) mCBContext->procs.hotplug = &hook_hotplug; // hotplug hook else mCBContext->procs.hotplug = NULL; memset(mCBContext->procs.zero, 0, sizeof(mCBContext->procs.zero)); mHwc->registerProcs(mHwc, &mCBContext->procs); // 真正注册 } // don't need a vsync thread if we have a hardware composer needVSyncThread = false; // 因为hardware composer存在,VSync由它来trigger,在SurfaceFlinger服务进程无需自己创建Vsync线程 // always turn vsync off when we start eventControl(HWC_DISPLAY_PRIMARY, HWC_EVENT_VSYNC, 0); // the number of displays we actually have depends on the // hw composer version if (hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_2)) { // 1.2 adds support for virtual displays mNumDisplays = MAX_DISPLAYS; } else if (hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1)) { // 1.1 adds support for multiple displays mNumDisplays = HWC_NUM_DISPLAY_TYPES; } else { mNumDisplays = 1; } } // 从gralloc模块获取一些显示输出相关信息 if (mFbDev) { ALOG_ASSERT(!(mHwc && hwcHasApiVersion(mHwc, HWC_DEVICE_API_VERSION_1_1)), "should only have fbdev if no hwc or hwc is 1.0"); DisplayData& disp(mDisplayData[HWC_DISPLAY_PRIMARY]); disp.connected = true; disp.width = mFbDev->width; disp.height = mFbDev->height; disp.format = mFbDev->format; disp.xdpi = mFbDev->xdpi; disp.ydpi = mFbDev->ydpi; if (disp.refresh == 0) { disp.refresh = nsecs_t(1e9 / mFbDev->fps); ALOGW("getting VSYNC period from fb HAL: %lld", disp.refresh); } if (disp.refresh == 0) { disp.refresh = nsecs_t(1e9 / 60.0); ALOGW("getting VSYNC period from thin air: %lld", mDisplayData[HWC_DISPLAY_PRIMARY].refresh); } } else if (mHwc) { // here we're guaranteed to have at least HWC 1.1 for (size_t i =0 ; i<HWC_NUM_DISPLAY_TYPES ; i++) { queryDisplayProperties(i); } } if (needVSyncThread) { // 如果Hardware composer不存在,则需要在SurfaceFlinger进程中创建VSync线程 // we don't have VSYNC support, we need to fake it mVSyncThread = new VSyncThread(*this); } }
// initialize the config and context
EGLint format = mHwc->getVisualID(); // 从hardware composer取到颜色空间
mEGLConfig = selectEGLConfig(mEGLDisplay, format); // 生成EGLConfig
mEGLContext = createGLContext(mEGLDisplay, mEGLConfig); //生成EGLContext(通过调用egl库的eglCreateContext函数,然后封装成egl_context_t对象)
// initialize our non-virtual displays for (size_t i=0 ; i<DisplayDevice::NUM_DISPLAY_TYPES ; i++) { DisplayDevice::DisplayType type((DisplayDevice::DisplayType)i); mDefaultDisplays[i] = new BBinder(); wp<IBinder> token = mDefaultDisplays[i]; // set-up the displays that are already connected if (mHwc->isConnected(i) || type==DisplayDevice::DISPLAY_PRIMARY) { // All non-virtual displays are currently considered secure. bool isSecure = true; mCurrentState.displays.add(token, DisplayDeviceState(type)); sp<FramebufferSurface> fbs = new FramebufferSurface(*mHwc, i); sp<SurfaceTextureClient> stc = new SurfaceTextureClient( static_cast< sp<ISurfaceTexture> >(fbs->getBufferQueue())); sp<DisplayDevice> hw = new DisplayDevice(this, type, isSecure, token, stc, fbs, mEGLConfig); if (i > DisplayDevice::DISPLAY_PRIMARY) { // FIXME: currently we don't get blank/unblank requests // for displays other than the main display, so we always // assume a connected display is unblanked. ALOGD("marking display %d as acquired/unblanked", i); hw->acquireScreen(); } mDisplays.add(token, hw); } }
// initialize OpenGL ES DisplayDevice::makeCurrent(mEGLDisplay, hw, mEGLContext); initializeGL(mEGLDisplay);
bool SurfaceFlinger::threadLoop() { waitForEvent(); // 通过Looper等待事件 return true; }
Android中创建创建一个Activity时创建Surface的流程:
● Activity Thread calls on attach() and makeVisible()
● makeVisible does wm.addView()
● vm.addView() - this also called by StatusBar to display itself
● Creates a new ViewRootImpl
● Call on its setView()
● setView() calls on sWindowSession.add(...)
● This results in call to WM's addWindow()
● ViewRootImpl's performTraversals()
● Calls on relayoutWindow()
● Calls to WM session's relayout()
● Call to WM's relayoutWindow()
● Call to createSurfaceLocked()
● new Surface(...) // Surface Object @java-layer
Surface构造函数调用了一个nativeCreate的native方法,其实现位于frameworks/base/core/jni/android_view_Surface.cpp:
static void nativeCreate(JNIEnv* env, jobject surfaceObj, jobject sessionObj, jstring nameStr, jint w, jint h, jint format, jint flags) { ScopedUtfChars name(env, nameStr); sp<SurfaceComposerClient> client(android_view_SurfaceSession_getClient(env, sessionObj)); sp<SurfaceControl> surface = client->createSurface( String8(name.c_str()), w, h, format, flags); if (surface == NULL) { jniThrowException(env, OutOfResourcesException, NULL); return; } setSurfaceControl(env, surfaceObj, surface); }
与此类似,接下来粘贴一段Native层创建Surface的Sample代码:
mComposerClient = new SurfaceComposerClient; ASSERT_EQ(NO_ERROR, mComposerClient->initCheck()); sp<IBinder> display(SurfaceComposerClient::getBuiltInDisplay( ISurfaceComposer::eDisplayIdMain)); DisplayInfo info; SurfaceComposerClient::getDisplayInfo(display, &info); // 获取显示参数 ssize_t displayWidth = info.w; ssize_t displayHeight = info.h; // Background surface mBGSurfaceControl = mComposerClient->createSurface( // 创建Surface,后文将详细分析其squence String8("BG Test Surface"), displayWidth, displayHeight, PIXEL_FORMAT_RGBA_8888, 0); ASSERT_TRUE(mBGSurfaceControl != NULL); ASSERT_TRUE(mBGSurfaceControl->isValid()); fillSurfaceRGBA8(mBGSurfaceControl, 63, 63, 195); // Fill an RGBA_8888 formatted surface with a single color. static void fillSurfaceRGBA8(const sp<SurfaceControl>& sc, uint8_t r, uint8_t g, uint8_t b) { Surface::SurfaceInfo info; sp<Surface> s = sc->getSurface(); ASSERT_TRUE(s != NULL); ASSERT_EQ(NO_ERROR, s->lock(&info)); uint8_t* img = reinterpret_cast<uint8_t*>(info.bits); for (uint32_t y = 0; y < info.h; y++) { for (uint32_t x = 0; x < info.w; x++) { uint8_t* pixel = img + (4 * (y*info.s + x)); pixel[0] = r; pixel[1] = g; pixel[2] = b; pixel[3] = 255; } } ASSERT_EQ(NO_ERROR, s->unlockAndPost()); }
queueBuffer的过程时序图如下:
VSync的时序图:
在dequeue了新的GraphicBuffer之后,SurfaceFlinger就需要对它管理的layer进行合成。合成的过程主要包含了以下几个重要的步骤:
handleMessageTransaction:处理系统显示屏以及应用程序窗口的属性变化。并把SurfaceFlinger::mDrawingStat更新为SurfaceFlinger::mCurrentState,新创建的layer(surface)都是保存在mCurrentState中的。
handlePageFlip:更新每个layer的active buffer。
rebuildLayerStacks:为每个DisplayDevice设置可见、按Z-order排序的layers
setUpHWComposer:根据在step3设置的SurfaceFlinger的DisplayDevice中的active layers来设置HWComposer的DisplayData数据,然后调用hwc模块的prepare函数
doComposition:使用OpenGL ES或者hwc模块来合成
在合成完后,就需要推送到屏幕上显示。有3个重要的类用来管理显示设备:
Android系统支持多个显示设备,每一个显示设备在SurfaceFlinger进程中都有一个DisplayDevice对象来管理它。
FramebufferSurface继承于ConsumerBase,当其连接的BufferQueue有新帧时,其onFrameAvailable方法会被调用,来处理这个新帧。
SurfaceTextureClient继承于ANativeWindow,在SurfaceFlinger进程中,它引用了FramebufferSurface的BufferQueue;同时又因为它是一个ANativeWindow,它被EGL封装为EGLSurface保存在DisplayDevice 中,用来显示改设备的新帧。
DisplayDevice的初始化过程在SurfaceFlinger::readyToRun()里实现。
下面重点介绍两种不同情况下的显示流程:
有HWC模块时: