Android VirtualDisplay解析

一.概述

       我们知道,Android设备都有录屏功能,其实主要是通过VirtualDisplay来实现的,VirtualDisplay对应虚拟Display,主要用来进行屏幕录制等相关功能;
       在DMS的registerDefaultDisplayAdapters()内部,除了创建LocalDisplayAdapter,还创建了VirtualDisplayAdapter,可以参考文章Android Display管理服务DMS,接下来一起看一下VirtualDisplayAdapter类实现:

二.VirtualDisplayAdapter

public VirtualDisplayAdapter(DisplayManagerService.SyncRoot syncRoot,
            Context context, Handler handler, Listener listener) {
    this(syncRoot, context, handler, listener,
                (String name, boolean secure) -> SurfaceControl.createDisplay(name, secure));
}

@VisibleForTesting
VirtualDisplayAdapter(DisplayManagerService.SyncRoot syncRoot,Context context, Handler handler, Listener listener,
            SurfaceControlDisplayFactory surfaceControlDisplayFactory) {
    super(syncRoot, context, handler, listener, TAG);
    mHandler = handler;
    mSurfaceControlDisplayFactory = surfaceControlDisplayFactory;
}

       在构造方法内部会通过SurfaceControl的createDisplay()获取的对象赋值给mSurfaceControlDisplayFactory变量,后续在创建VirtualDisplayDevice时需要;

三.VirtualDisplayDevice

       通过VirtualDisplayAdapter内部的createVirtualDisplayLocked()来创建VirtualDisplayDevice;

public DisplayDevice createVirtualDisplayLocked(IVirtualDisplayCallback callback,
            IMediaProjection projection, int ownerUid, String ownerPackageName, String name,
            int width, int height, int densityDpi, Surface surface, int flags, String uniqueId) {
    boolean secure = (flags & VIRTUAL_DISPLAY_FLAG_SECURE) != 0;
    IBinder appToken = callback.asBinder();
    IBinder displayToken = mSurfaceControlDisplayFactory.createDisplay(name, secure);
    final String baseUniqueId =
                UNIQUE_ID_PREFIX + ownerPackageName + "," + ownerUid + "," + name + ",";
    final int uniqueIndex = getNextUniqueIndex(baseUniqueId);
    if (uniqueId == null) {
        uniqueId = baseUniqueId + uniqueIndex;
    } else {
        uniqueId = UNIQUE_ID_PREFIX + ownerPackageName + ":" + uniqueId;
    }
    VirtualDisplayDevice device = new VirtualDisplayDevice(displayToken, appToken,
                ownerUid, ownerPackageName, name, width, height, densityDpi, surface, flags,
                new Callback(callback, mHandler), uniqueId, uniqueIndex);

    mVirtualDisplayDevices.put(appToken, device);
    try {
        if (projection != null) {
            projection.registerCallback(new MediaProjectionCallback(appToken));
        }
        appToken.linkToDeath(device, 0);
    }
    return device;
}

       1.通过mSurfaceControlDisplayFactory的createDisplay()来获取对应的displayToken,由于是虚拟设备,跟BUILT_IN设备获取方式是不同的;
       2.VirtualDisplay的mUniqueId格式为:virtual:com.hly.test,1000(ownerUid),-display(name),0;
       3.将创建的VirtualDisplayDevice加入mVirtualDisplayDevices进行管理;
       4.projection.registerCallback()执行录屏相关注册;

四.VirtualDisplay创建

       当要执行屏幕录制前,会先通过MediaProjection执行createVirtualDisplay()来创建VirtualDisplay;

1.createVirtualDisplay()

public VirtualDisplay createVirtualDisplay(@NonNull String name,int width, int height, int dpi, int flags, @Nullable Surface surface,@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
     DisplayManager dm = (DisplayManager) mContext.getSystemService(Context.DISPLAY_SERVICE);
     return dm.createVirtualDisplay(this, name, width, height, dpi, surface, flags, callback,handler, null /* uniqueId */);
}

       需要传入VirtualDisplay的name、width、height、Surface等,Surface用来获取或显示屏幕数据,最后会调用到DisplayMangerGlobal内部的createVirtualDisplay()方法:

2.createVirtualDisplay()

public VirtualDisplay createVirtualDisplay(Context context, MediaProjection projection,
            String name, int width, int height, int densityDpi, Surface surface, int flags,
            VirtualDisplay.Callback callback, Handler handler, String uniqueId) {
    ..........................
    VirtualDisplayCallback callbackWrapper = new VirtualDisplayCallback(callback, handler);
    IMediaProjection projectionToken = projection != null ? projection.getProjection() : null;
    int displayId;
    try {
        displayId = mDm.createVirtualDisplay(callbackWrapper, projectionToken,
                    context.getPackageName(), name, width, height, densityDpi, surface, flags,
                    uniqueId);
    } catch (RemoteException ex) {
        throw ex.rethrowFromSystemServer();
    }
    Display display = getRealDisplay(displayId);
    ...............................
    return new VirtualDisplay(this, display, callbackWrapper, surface);
}

       接下来会调用执行到DMS的createVirtualDisplay()来先创建VirtualDisplayDevice和LogicalDisplay,根据LogicalDisplay的displayId创建Display,然后将Display作为参数创建VirtualDisplay返回;

3.总结

       用一张流程图总结一下创建流程:


image.png

五.镜像实现

       在录屏时,其实就是一个镜像,VirtualDisplayDevice是没有内容显示的,所以需要将其镜像到要录制屏对应的DisplayDevice就可以了,具体实现逻辑是在DMS的configureDisplayInTransactionLocked()方法,那该方法是什么时候会触发调用呢?
       根据前面的分析,在创建VirtualDisplay时,会执行到handleDisplayDeviceAddedLocked(),一起看一下该方法:

1.handleDisplayDeviceAddedLocked()

private void handleDisplayDeviceAddedLocked(DisplayDevice device) {
    DisplayDeviceInfo info = device.getDisplayDeviceInfoLocked();
    .....................
    mDisplayDevices.add(device);
    LogicalDisplay display = addLogicalDisplayLocked(device);
    Runnable work = updateDisplayStateLocked(device);
    if(work != null) {
        work.run();
    }
    scheduleTraversalLocked(false);
}

       在该方法内部会创建VirtualDisplayDevice对应的LogicalDisplay,继而分配displayId及layerStack,那创建完后是如何跟目标屏建立对应关系的呢?关键点就在scheduleTraversalLocked(false),一起看一下:

2.scheduleTraversalLocked()

private void scheduleTraversalLocked(boolean inTraversal) {
    if (!mPendingTraversal && mWindowManagerInternal != null) {
        mPendingTraversal = true;
        if (!inTraversal) {
            mHandler.sendEmptyMessage(MSG_REQUEST_TRAVERSAL);
        }
    }
}

case MSG_REQUEST_TRAVERSAL:
     mWindowManagerInternal.requestTraversalFromDisplayManager();
     break;

private final class LocalService extends WindowManagerInternal {
    @Override
     public void requestTraversalFromDisplayManager() {
        requestTraversal();
     }
}

void requestTraversal() {
    synchronized (mWindowMap) {
        mWindowPlacerLocked.requestTraversal();
    }
}

       在scheduleTraversalLocked()内部发送 MSG_REQUEST_TRAVERSAL消息,执行该消息的时候就是调用WMS的LocalService执行 requestTraversal()方法,具体执行过程简单罗列一下:

WindowSurfacePlacer -> requestTraversal()
 WindowSurfacePlacer -> performSurfacePlacement()
  WindowSurfacePlacer -> performSurfacePlacementLoop()
   RootWindowContainer -> performSurfacePlacement()
    void performSurfacePlacement(boolean recoveringMemory) {
       ...................
       mService.openSurfaceTransaction();
        try {
            applySurfaceChangesTransaction(recoveringMemory, defaultDw, defaultDh);
        } catch (RuntimeException e) {
            Slog.wtf(TAG, "Unhandled exception in Window Manager", e);
        } finally {
            mService.closeSurfaceTransaction();
        }
      ......
    }

       可以看到,在RootWindowContainer内部的performSurfacePlacement()会顺序执行以下方法:
       1.mService.openSurfaceTransaction(),通过SurfaceControl来通知native开始一个Transaction;
       2.applySurfaceChangesTransaction(recoveringMemory, defaultDw, defaultDh)来处理Transaction;
       3.mService.closeSurfaceTransaction(),通过SurfaceControl来通知native(SurfaceFlinger)关闭一个Transaction最终来执行合成显示等工作;

3.applySurfaceChangesTransaction()

private void applySurfaceChangesTransaction(boolean recoveringMemory, int defaultDw,
            int defaultDh) {
    ......................................
    // Give the display manager a chance to adjust properties like display rotation if it needs
    // to.
    mService.mDisplayManagerInternal.performTraversalInTransactionFromWindowManager();
}

       刷新后经过一番调用,最终回到DisplayManagerService的performTraversalInTransactionFromWindowManager()方法,该方法会调用到performTraversalInTransactionLocked()方法:

4.performTraversalInTransactionLocked()

private void performTraversalInTransactionLocked() {
    .....................
    // Configure each display device.
    final int count = mDisplayDevices.size();
    for (int i = 0; i < count; i++) {
        DisplayDevice device = mDisplayDevices.get(i);
        configureDisplayInTransactionLocked(device);
        device.performTraversalInTransactionLocked();
    }
    ......................
}

       可以看到,在该方法内会遍历所有的DisplayDevice来主要干了两件事:
       1.执行configureDisplayInTransactionLocked()进行配置DisplayDevice要显示的layerStack;
       2.执行performTraversalInTransactionLocked()来关联Surface,即数据显示源;
       先看configureDisplayInTransactionLocked():

5.configureDisplayInTransactionLocked()

private void configureDisplayInTransactionLocked(DisplayDevice device) {
    final DisplayDeviceInfo info = device.getDisplayDeviceInfoLocked();
    final boolean ownContent = (info.flags & DisplayDeviceInfo.FLAG_OWN_CONTENT_ONLY) != 0;
    LogicalDisplay display = findLogicalDisplayForDeviceLocked(device);

    if (!ownContent) {
        if (display != null && !display.hasContentLocked()) {
            // If the display does not have any content of its own, then
            // automatically mirror the default logical display contents.
            display = null;
        }
        if (display == null) {
            display = mLogicalDisplays.get(Display.DEFAULT_DISPLAY);
        }
    }

    display.configureDisplayInTransactionLocked(device, info.state == Display.STATE_OFF);
    ....................
}

       当DisplayDevice对应VirtualDisplayDevice时,返回的ownContent为false,即没有自己内容显示的,进入判断,通过display.hasContentLocked()返回false,所以得到的display为null,接下来如果display为null,会从mLogicalDisplays中取出DEFAULT_DISPLAY赋值给display,那有什么用呢?
       关键逻辑在display.configureDisplayInTransactionLocked():

6.configureDisplayInTransactionLocked()

public void configureDisplayInTransactionLocked(DisplayDevice device, boolean isBlanked) {
    // Set the layer stack.
    device.setLayerStackInTransactionLocked(isBlanked ? BLANK_LAYER_STACK : mLayerStack);
    .....................
    .....................
}

public final void setLayerStackInTransactionLocked(int layerStack) {
    if (mCurrentLayerStack != layerStack) {
        mCurrentLayerStack = layerStack;
        SurfaceControl.setDisplayLayerStack(mDisplayToken, layerStack);
    }
}

       在该方法内部执行device.setLayerStackInTransactionLocked(),即将VirtualDisplayDevice的layerStack设置为DEFAULT_DISPLAY对应的layerStack,继而调用SurfaceControl内部的setDisplayLayerStack()方法,mDisplayToken用于唯一标识SurfaceFliger创建的VirtualDisplay,跟随调用关系,再一起看一下:

public static void setDisplayLayerStack(IBinder displayToken, int layerStack) {
    nativeSetDisplayLayerStack(displayToken, layerStack);
}

static void nativeSetDisplayLayerStack(JNIEnv* env, jclass clazz,jobject tokenObj, jint layerStack) {
    sp token(ibinderForJavaObject(env, tokenObj));
    SurfaceComposerClient::setDisplayLayerStack(token, layerStack);
}

void SurfaceComposerClient::setDisplayLayerStack(const sp& token,uint32_t layerStack) {
    Composer::getInstance().setDisplayLayerStack(token, layerStack);
}

void Composer::setDisplayLayerStack(const sp& token,uint32_t layerStack) {
    Mutex::Autolock _l(mLock);
    DisplayState& s(getDisplayStateLocked(token));
    s.layerStack = layerStack;
    s.what |= DisplayState::eLayerStackChanged;
}

       跟随调用关系,主要有以下几个要点:
       getDisplayStateLocked():如果不存在该token对应的 DisplayState就会为该token创建一个DisplayState,并保存在mDisplayStates中进行管理;
       接下来将layerStack赋值给DisplayyState的layerStack变量,此时layerStack配置完成;
       前面讲到,在创建VirtualDisplay时会传入Surface,那该Surface是如何与VirtualDisplay进行关联的呢?关键在performTraversalInTransactionLocked()方法:

7.performTraversalInTransactionLocked()

@Override
public void performTraversalInTransactionLocked() {
    if ((mPendingChanges & PENDING_RESIZE) != 0) {
         SurfaceControl.setDisplaySize(getDisplayTokenLocked(), mWidth, mHeight);
    }
    if ((mPendingChanges & PENDING_SURFACE_CHANGE) != 0) {
        setSurfaceInTransactionLocked(mSurface);
    }
    mPendingChanges = 0;
}

public final void setSurfaceInTransactionLocked(Surface surface) {
    if (mCurrentSurface != surface) {
        mCurrentSurface = surface;
        SurfaceControl.setDisplaySurface(mDisplayToken, surface);
    }
}

       mSurface是在创建VirtualDisplay时传入的,可以是通过SurfaceView的getSurface()获取的Surface,用来直接显示;也可以是通过MediaCodec获取的Surface,用来实现远端视频流传输;

public static void setDisplaySurface(IBinder displayToken, Surface surface) {
    ..........
    if (surface != null) {
        synchronized (surface.mLock) {
            nativeSetDisplaySurface(displayToken, surface.mNativeObject);
        }
    } else {
        nativeSetDisplaySurface(displayToken, 0);
    }
}

static void nativeSetDisplaySurface(JNIEnv* env, jclass clazz,jobject tokenObj, jlong nativeSurfaceObject) {
    sp token(ibinderForJavaObject(env, tokenObj));
    sp bufferProducer;
    sp sur(reinterpret_cast(nativeSurfaceObject));
    if (sur != NULL) {
        bufferProducer = sur->getIGraphicBufferProducer();
    }
    status_t err = SurfaceComposerClient::setDisplaySurface(token,
        bufferProducer);
    }
}

       bufferProducer主要用于申请 Buffer,申请Buffer是写入图形数据;consumer主要用于消费Buffer,消费Buffer就是把应用写入数据准备好的Buffer取出来处理;

status_t SurfaceComposerClient::setDisplaySurface(const sp& token,sp bufferProducer) {
    return Composer::getInstance().setDisplaySurface(token, bufferProducer);
}

status_t Composer::setDisplaySurface(const sp& token,sp bufferProducer) {
    DisplayState& s(getDisplayStateLocked(token));
    s.surface = bufferProducer;
    s.what |= DisplayState::eSurfaceChanged;
    return NO_ERROR;
}

       把 bufferProducer 保存到 DisplayState 的 surface中;
       经过上面的处理后,为VirtualDisplay配置好了 layerStack (镜像屏)和bufferProducer,那么接下来是如何进行后续显示处理的呢?
       前面讲到,在RootWindowContainer内部的performSurfacePlacement()在执行applySurfaceChangesTransaction()完后会执行mService.closeSurfaceTransaction(),一起看一下该方法:

8.closeSurfaceTransaction()

void closeSurfaceTransaction() {
    try {
        synchronized (mWindowMap) {
            if (mRoot.mSurfaceTraceEnabled) {
                mRoot.mRemoteEventTrace.closeSurfaceTransaction();
            }
            SurfaceControl.closeTransaction();
        }
    } finally {
    }
}

public static void closeTransaction() {
    nativeCloseTransaction(false);
}

static void nativeCloseTransaction(JNIEnv* env, jclass clazz, jboolean sync) {
    SurfaceComposerClient::closeGlobalTransaction(sync);
}

void SurfaceComposerClient::closeGlobalTransaction(bool synchronous) {
    Composer::closeGlobalTransaction(synchronous);
}

void Composer::closeGlobalTransactionImpl(bool synchronous) {
    sp sm(ComposerService::getComposerService());
    Vector transaction;
    Vector displayTransaction;
    ......
    {
        displayTransaction = mDisplayStates;
    }
     ......
   sm->setTransactionState(transaction, displayTransaction, flags); //sm  SurfaceFlinger
}

       sm:指向SurfaceFlinger 代理对象;mDisplayStates:里面保存了之前创建的VirtualDisplay对应的DisplayState,在DisplayState中保存了layerStack(镜像屏)和 bufferProducer(写入屏幕数据);

9.setTransactionState()

void SurfaceFlinger::setTransactionState(const Vector& state,const Vector& displays,uint32_t flags){
    .....
    size_t count = displays.size();
    for (size_t i=0 ; i

       1.遍历mDisplayStates执行setDisplayStateLocked()更新 mCurrentState.displays 中的数据,通过token取出 VirtualDisplay然后更新surface和layerStack;

uint32_t SurfaceFlinger::setDisplayStateLocked(const DisplayState& s){
     ssize_t dpyIdx = mCurrentState.displays.indexOfKey(s.token);
     ......
     DisplayDeviceState& disp(mCurrentState.displays.editValueAt(dpyIdx));
     if (disp.isValid()) {
         const uint32_t what = s.what;
         if (what & DisplayState::eSurfaceChanged) {
            if (IInterface::asBinder(disp.surface) != IInterface::asBinder(s.surface)) {
                disp.surface = s.surface;
                flags |= eDisplayTransactionNeeded;
            }
         }
         if (what & DisplayState::eLayerStackChanged) {
             if (disp.layerStack != s.layerStack) {
                 disp.layerStack = s.layerStack;
                 flags |= eDisplayTransactionNeeded;
             }
         }
         ......
     }
     return flags;
}

       2.通过setTransactionFlags()发起一次刷新;

uint32_t SurfaceFlinger::setTransactionFlags(uint32_t flags) {
    signalTransaction();
}

void SurfaceFlinger::signalTransaction() {
    mEventQueue.invalidate();
}

void MessageQueue::invalidate() {
    mEvents->requestNextVsync();
}

       SurfaceFlinger 请求下一个同步帧,执行一次刷新,最终调用如下: SurfaceFlinger::handleTransaction()-->SurfaceFlinger::handleTransactionLocked();

10.handleTransactionLocked()

void SurfaceFlinger::handleTransactionLocked(uint32_t transactionFlags){
    .......................
    .......................
    const KeyedVector<  wp, DisplayDeviceState>& curr(mCurrentState.displays);
    const KeyedVector<  wp, DisplayDeviceState>& draw(mDrawingState.displays);
    const size_t cc = curr.size();
    size_t dc = draw.size();
    for (size_t i=0 ; iisUsingVrComposer()) {
                      .....
                      displayUtils->initVDSInstance(*mHwc, hwcId, state.surface,dispSurface, producer, bqProducer, bqConsumer,state.displayName, state.isSecure);
                  }
               }
          }
          ......
          if (dispSurface != NULL && producer != NULL) {
              sp hw =new DisplayDevice(this, state.type, hwcId, state.isSecure, display,
                                                  dispSurface, producer,mRenderEngine->getEGLConfig(),
                                                  hasWideColorDisplay);
              hw->setLayerStack(state.layerStack);
              hw->setProjection(state.orientation,state.viewport, state.frame);
              hw->setDisplayName(state.displayName);
              .......
              mDisplays.add(display, hw);
              if (!state.isVirtualDisplay()) {
                  mEventThread->onHotplugReceived(state.type, true);
              }
         }
     }
}

       先看一下两个变量含义:
       mCurrentState:SurfaceFlinger中最新的状态,包括所有的display和所有要绘制的Layer;
       mDrawingState:SurfaceFlinger中当前显示的状态,包括所有的display和所有要绘制的Layer;
       通过 mCurrentState 和 mDrawingState 比较是否有增加的display,如果有增加的dislay并且是 VirtualDisplay,首先会通过 initVDSInstance 函数创建一个 VirtualDisplaySurface封装了producer、consumer以及surface,然后创建一个 DisplayDevice并设置其 layerStack,最后保存到 mDisplays 中进行管理,此时虚拟屏幕的 DisplayDevice创建好了。
       在SurfaceFliner刷新所有的Layer到屏幕上显示之前,需要确定那些Layer应该显示到哪个屏幕上,以上逻辑是通过rebuildLayerStacks()来进行实现的:

11.rebuildLayerStacks()

void SurfaceFlinger::rebuildLayerStacks() {
     ......
     for (size_t dpy=0 ; dpy> layersSortedByZ;
          Vector> layersNeedingFences;
          const sp& displayDevice(mDisplays[dpy]);
          const Transform& tr(displayDevice->getTransform());
          const Rect bounds(displayDevice->getBounds());
          if (displayDevice->isDisplayOn()) {
              computeVisibleRegions(displayDevice, dirtyRegion, opaqueRegion);
              mDrawingState.traverseInZOrder([&](Layer* layer) {
                 if (layer->belongsToDisplay(displayDevice->getLayerStack(),displayDevice->isPrimary())) {
                      ......................
                      layersSortedByZ.add(layer);
                      ...........
                    });
                }
                displayDevice->setVisibleLayersSortedByZ(layersSortedByZ);
                ......
        }
    }
}

bool belongsToDisplay(uint32_t layerStack, bool isPrimaryDisplay) const {
     return getLayerStack() == layerStack && (!mPrimaryDisplayOnly || isPrimaryDisplay);
}

       在该方法内部会遍历mDisplays来将要显示的layer分配到对应的displayDevice上,主要有以下几个变量:
       Layer:对应WMS中的WindowState;
       layersSortedByZ:保存Layer的Vector;
       displayDevice->getLayerStack():创建VirtualDisplayDevice时为VirtualDisplay分配的LayerStack;
       belongsToDisplay():判断Layer的layerStack是否和 displayDevice的LayerStack一致,如果一致是就把该Layer加入到layersSortedByZ中保存;
       最后把所有要显示到该display上的Layer (layersSortedByZ) 保存到 displayDevice 中;
       总结一下:前面创建的VirtualDisplay的layerStack 对应就是主屏的layerStack,也就是说,此方法会把所有要显示到主屏的layer全部都保存到VirtualDisplay 中,这样VirtualDisplay就拿到了所有要显示到主屏的内容。
       关于录屏实现,可以参考文章Android 屏幕直播分享之MediaProjection和MediaCodec分析;

你可能感兴趣的:(Android VirtualDisplay解析)