Camera学习心得(一) open流程

简述

接触Camera大概一年多时间了，因为平时工作侧重点主要是解决客户需求，并没有对Camera整个流程有个很系统性的了解，趁着最近项目不是很忙，对整个流程大概梳理一下．

在Android L版本之前Camera一直采用的是API1+HAL1架构，Android L版本以后，推出了API2+HAL3架构，从官方介绍来看，这个改变主要是性能方面的提升以及支持一些新的特性，当然具体还要看厂商的对这块的支持程度．这些不是本文的重点，本文主要是对Camera API1的调用流程进行一个简单的梳理．

这是网上看到的一张关于Camera API1的流程框架图，其实Camera的层级结构是和Android架构对应的，自上而下，App层，Framework层，Runtime层，Library层，HAL层，首先从App层说起．

open流程

目前负责的Camera项目是基于MTK平台的Camera2来进行开发的，这套Camera代码结构设计的还是挺牛逼的，遵循高内聚，低耦合，试图分离，逻辑还是比较清晰的，主要有host,common,feature三个主要目录，host为主框架，包括了CameraActivity,CameraUI等主要的类；common里面主要包含一些公共的类库，定义了一些通信的公共接口；feature里面又包含了mode和settings两个目录，mode里面主要就是Camera里面的每一个模式的相关类,settings顾名思义就是Camera里面关于设置的相关类，有兴趣的可以自行查看源码．

PhotoDeviceController.java

doOpenCamera

    try {
                //when open the camera need reset the mCameraProxy to null
                if (sync) {
                    mCameraDeviceManager.openCameraSync(mCameraId, mCameraProxyStateCallback, null);
                } else {
                    mCameraDeviceManager.openCamera(mCameraId, mCameraProxyStateCallback, null);①
                }
            }  catch (CameraOpenException e) {
                //need show error and finish the activity.
                if (CameraOpenException.ExceptionType.SECURITY_EXCEPTION == e.getExceptionType()) {
                    CameraUtil.showErrorInfoAndFinish(mActivity, CameraUtil
                            .CAMERA_HARDWARE_EXCEPTION);
                }
        }

截取部分关键代码，①处执行了mCameraDeviceManager.openCamera方法，第一个参数就是我们要open的camera ID,后摄是０，前摄为１，第二个参数，是open camera的状态，如下面代码所示：

 /**
         * Open camera device state callback, this callback is send to camera device manager
         * by open camera interface.
         */
        private class CameraDeviceProxyStateCallback extends CameraProxy.StateCallback {

            @Override
            public void onOpened(@Nonnull CameraProxy cameraProxy) {
                LogHelper.i(TAG, "[onOpened]proxy = " + cameraProxy + " state = " + mCameraState);
                synchronized (mWaitCameraOpenDone) {
                    mCameraProxy = cameraProxy;
                    mWaitCameraOpenDone.notifyAll();
                    mRequestHandler.obtainMessage(PhotoDeviceAction.ON_CAMERA_OPENED, cameraProxy)
                            .sendToTarget();
                }
            }
           @Override
            public void onClosed(@Nonnull CameraProxy cameraProxy) {
                LogHelper.i(TAG, "[onClosed] current proxy : " + mCameraProxy + " closed proxy " +
                        "= " + cameraProxy);
                if (mCameraProxy != null && mCameraProxy == cameraProxy) {
                    synchronized (mWaitCameraOpenDone) {
                        mWaitCameraOpenDone.notifyAll();
                    }
                }
            }
            @Override
            public void onDisconnected(@Nonnull CameraProxy cameraProxy) {
                LogHelper.i(TAG, "[onDisconnected] current proxy : " + mCameraProxy + " closed " +
                        " proxy " + cameraProxy);
                mCaptureCount.set(0);
                captureDone();
                if (mCameraProxy != null && mCameraProxy == cameraProxy) {
                    synchronized (mWaitCameraOpenDone) {
                        mCameraState = CameraState.CAMERA_UNKNOWN;
                        mWaitCameraOpenDone.notifyAll();
                        mRequestHandler.obtainMessage(PhotoDeviceAction.ON_CAMERA_DISCONNECTED,
                                cameraProxy).sendToTarget();
                    }
                }
            }
            @Override
            public void onError(@Nonnull CameraProxy cameraProxy, int error) {
                LogHelper.i(TAG, "[onError] current proxy : " + mCameraProxy + " error " + error +
                        " proxy " + cameraProxy);
                //if current is in capturing, but close is wait capture done,
                //so this case need notify the capture done. otherwise will be ANR to pause.
                mCaptureCount.set(0);
                captureDone();
                if ((mCameraProxy != null && mCameraProxy == cameraProxy)
                        || error == CameraUtil.CAMERA_OPEN_FAIL) {
                    synchronized (mWaitCameraOpenDone) {
                        mCameraState = CameraState.CAMERA_UNKNOWN;
                        mWaitCameraOpenDone.notifyAll();
                        mRequestHandler.obtainMessage(PhotoDeviceAction.ON_CAMERA_ERROR, error, 0,
                                cameraProxy).sendToTarget();
                    }
                }
            }
        }
    }

从代码里面可以看出,open,close,disconnect,error状态都有相应的回调,最后一个参数直接传递null值,经过一些接口调用和消息传递，最终会执行到CameraDeviceManagerImpl.java这个类中，接着往下看．

CameraDeviceManagerImpl.java

public void handleMessage(Message msg) {
                super.handleMessage(msg);
                switch (msg.what) {
                    case MSG_OPEN_CAMERA:
                  ．．．
                        if (mCameraProxy == null) {
                            LogHelper.i(mHandlerTag, "[openCamera]+");
                            long startTime = SystemClock.uptimeMillis();
                            FeatureLoader.notifySettingBeforeOpenCamera(mContext, mCameraId,
                                    CameraApi.API1);
                            retryOpenCamera();①
                        ｝
                        mOpenStateCallback.onOpened(mCameraProxy)；②camera已经打开，可以执行接下来的动作startPreview...
                  ．．．
                }
            }

我们只看关键代码，①处代码调用了retryOpenCamera方法：

private void retryOpenCamera() {
                try {
                    mCamera = CameraEx.openLegacy(Integer.parseInt(mCameraId),
                            CameraEx.CAMERA_HAL_API_VERSION_1_0);①
                } catch (RuntimeException e) {
                    LogHelper.e(mHandlerTag, "[retryOpenCamera] error: " + e.getMessage());
                    if (isNeedRetryOpen()) {
                        mOpenStateCallback.onRetry();
                        LogHelper.e(mHandlerTag, "[retryOpenCamera] retry time: " + mRetryCount);
                        retryOpenCamera();
                        return;
                    }
                    mOpenStateCallback.onError(new CameraProxy(mCameraId, mRequestHandler, mCamera),
                            CameraUtil.CAMERA_OPEN_FAIL);
                }
            }
            private boolean isNeedRetryOpen() {
                if (mRetryCount < OPEN_RETRY_COUNT) {
                    mRetryCount++;
                    return true;
                }
                return false;
  }

在retryOpenCamera方法里面,调用了MTK封装的openLegacy方法,参数里面我们也可以看到，传递的是CAMERA_HAL_API_VERSION_1_0,说明走的是HAL1流程：

public static Camera openLegacy(int cameraId, int halVersion) {
        if (VERSION.SDK_INT > 21 && VERSION.SDK_INT < 28) {
            try {
                return Camera.openLegacy(cameraId, halVersion);
            } catch (RuntimeException var3) {
                Log.e("CamAp_CameraEx", "[openLegacy] exception:" + var3);
                throw var3;
            }
        } else {
            return Camera.open(cameraId);
        }
    }

在openLegacy方法里面,可以看到如果我们的Android版本高于5.0执行的是MTK封装的openLegacy方法，反之直接执行open方法,这里我们直接分析Camera.open方法．

Framework层

Camera.java

    public static Camera open() {
        int numberOfCameras = getNumberOfCameras();
        CameraInfo cameraInfo = new CameraInfo();
        for (int i = 0; i < numberOfCameras; i++) {
            getCameraInfo(i, cameraInfo);
            if (cameraInfo.facing == CameraInfo.CAMERA_FACING_BACK) {①
                return new Camera(i);②
            }
        }
        return null;
    }

首先获取摄像头的个数，通过getCameraInfo(i, cameraInfo)来获取Camera的相关信息，①处判断可以看到如果不指定打开的Camera ID的话，默认打开的是后置摄像头，②处直接返回一个Camera对象：

Camera(int cameraId) {
        int err = cameraInitNormal(cameraId);
        if (checkInitErrors(err)) {
            if (err == -EACCES) {
                throw new RuntimeException("Fail to connect to camera service");
            } else if (err == -ENODEV) {
                throw new RuntimeException("Camera initialization failed");
            }
            // Should never hit this.
            throw new RuntimeException("Unknown camera error");
        }
    }

private int cameraInitNormal(int cameraId) {
        return cameraInitVersion(cameraId, CAMERA_HAL_API_VERSION_NORMAL_CONNECT);
    }

private int cameraInitVersion(int cameraId, int halVersion) ｛
　　．．．
        return native_setup(new WeakReference(this), cameraId, halVersion,
                ActivityThread.currentOpPackageName());
       ．．．
    }

private native final int native_setup(Object camera_this, int cameraId, int halVersion,
                                           String packageName);

这里调用了native_setup方法，到此，Framework层调用流程结束．

Runtime层

android_hardware_Camera.cpp

static jint android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz,
    jobject weak_this, jint cameraId, jint halVersion, jstring clientPackageName)
{
  ...
    sp camera;
    if (halVersion == CAMERA_HAL_API_VERSION_NORMAL_CONNECT) {
        // Default path: hal version is don't care, do normal camera connect.
        camera = Camera::connect(cameraId, clientName,
                Camera::USE_CALLING_UID, Camera::USE_CALLING_PID);①
    } else {
        jint status = Camera::connectLegacy(cameraId, halVersion, clientName,
                Camera::USE_CALLING_UID, camera);
        if (status != NO_ERROR) {
            return status;
        }
    }
    ...
}

这里注册了native_setup本地方法，有关JNI的相关知识可以自行了解，这里就不在具体说明，sp代表枪强指针，①处代码可以看到调用了Camera.cpp的connect方法．

Library层

Library层采用了C/S架构，分别为Client,Service,先来看一下Client的主要流程：

Client

Camera.cpp

sp Camera::connect(int cameraId, const String16& clientPackageName,
        int clientUid, int clientPid)
{
    return CameraBaseT::connect(cameraId, clientPackageName, clientUid, clientPid);
}

这里调用了CameraBaseT的connect函数，继续看下CameraBase.h这个头文件里面的定义：

CameraBase.h

 typedef CameraBase         CameraBaseT;

所以CameraBaseT::connect实际调用的是CameraBase.cpp里面的connect函数：

CameraBase.cpp

sp CameraBase::connect(int cameraId,
                                               const String16& clientPackageName,
                                               int clientUid, int clientPid)
{
    ALOGV("%s: connect", __FUNCTION__);
    sp c = new TCam(cameraId);
    sp cl = c;
    const sp<::android::hardware::ICameraService> cs = getCameraService();①

    binder::Status ret;
    if (cs != nullptr) {
        TCamConnectService fnConnectService = TCamTraits::fnConnectService;
        ret = (cs.get()->*fnConnectService)(cl, cameraId, clientPackageName, clientUid,
                                               clientPid, /*out*/ &c->mCamera);②
    }
    if (ret.isOk() && c->mCamera != nullptr) {
        IInterface::asBinder(c->mCamera)->linkToDeath(c);
        c->mStatus = NO_ERROR;
    } else {
        ALOGW("An error occurred while connecting to camera %d: %s", cameraId,
                (cs == nullptr) ? "Service not available" : ret.toString8().string());
        c.clear();
    }
    return c;
}

这里面的代码就关键了，先来看下①处代码，这里调用了getService函数：

const sp<::android::hardware::ICameraService> CameraBase::getCameraService()
{
    Mutex::Autolock _l(gLock);
    if (gCameraService.get() == 0) {
        char value[PROPERTY_VALUE_MAX];
        property_get("config.disable_cameraservice", value, "0");
        if (strncmp(value, "0", 2) != 0 && strncasecmp(value, "false", 6) != 0) {
            return gCameraService;
        }

        sp sm = defaultServiceManager();①
        sp binder;
        do {
            binder = sm->getService(String16(kCameraServiceName));
            if (binder != 0) {
                break;
            }
            ALOGW("CameraService not published, waiting...");
            usleep(kCameraServicePollDelay);
        } while(true);
        if (gDeathNotifier == NULL) {
            gDeathNotifier = new DeathNotifier();
        }
        binder->linkToDeath(gDeathNotifier);
        gCameraService = interface_cast<::android::hardware::ICameraService>(binder);②
    }
    ALOGE_IF(gCameraService == 0, "no CameraService!?");
    return gCameraService;
}

上面已经提到过，Library层采用的C/S架构，分为Client process和Service process,两个进程之间通过什么进行通信呢，自然是Binder,BInder在Native层的原理暂时还没有搞得太明白，接下来准备深入研究一下这块，这里只简单的说一下.首先判断gCameraService(ICameraService对象)是否已经存在,如果已经存在则直接返回,如果不存在则执行①处代码,①处的函数在
IServiceManager.cpp里面定义：

Server

IServiceManager.cpp

sp defaultServiceManager()
{
    if (gDefaultServiceManager != NULL) return gDefaultServiceManager;
    {
        AutoMutex _l(gDefaultServiceManagerLock);
        while (gDefaultServiceManager == NULL) {
            gDefaultServiceManager = interface_cast(
                ProcessState::self()->getContextObject(NULL));
            if (gDefaultServiceManager == NULL)
                sleep(1);
        }
    }
    return gDefaultServiceManager;
}

可以看到这里返回的IServiceManager类型的gDefaultServiceManager对象,回到getCameraService函数中,获取到gDefaultServiceManager对象后,通过调用getService函数来获取ICameraService对象,看下fnConnectService如何定义(在Camera.cpp中定义)的：

CameraTraits::TCamConnectService CameraTraits::fnConnectService =
        &::android::hardware::ICameraService::connect;

所以最终还是调用了ICameraService的connect函数，也就是CameraService.cpp的connect函数：

CameraService.cpp

Status CameraService::connect(
        const sp& cameraClient,
        int cameraId,
        const String16& clientPackageName,
        int clientUid,
        int clientPid,
        /*out*/
        sp* device) {

    ATRACE_CALL();
    Status ret = Status::ok();
    String8 id = String8::format("%d", cameraId);
    sp client = nullptr;
#if 0
    ret = connectHelper(cameraClient, id,
            CAMERA_HAL_API_VERSION_UNSPECIFIED, clientPackageName, clientUid, clientPid, API_1,
            /*legacyMode*/ false, /*shimUpdateOnly*/ false,
            /*out*/client);
#else // Third-party using API1.0, add by [email protected] at 20180502 start +++
    ret = connectHelper(cameraClient, id,
            256/*CAMERA_HAL_API_VERSION_1_0*/, clientPackageName, clientUid, clientPid, API_1,
            /*legacyMode*/ false, /*shimUpdateOnly*/ false,
            /*out*/client);
#endif // end +++

    if(!ret.isOk()) {
        logRejected(id, getCallingPid(), String8(clientPackageName),
                ret.toString8());
        return ret;
    }

    *device = client;
    return ret;
}

可以看到在connect函数里面调用了connectHelper函数：

Status CameraService::connectHelper(const sp& cameraCb, const String8& cameraID,
        int halVersion, const String16& clientPackageName, int clientUid, int clientPid,
        apiLevel effectiveApiLevel, bool legacyMode, bool shimUpdateOnly,
        /*out*/sp& device) 
        ．．．
        if(!(ret = makeClient(this, cameraCb, clientPackageName, cameraId, facing, clientPid,
                clientUid, getpid(), legacyMode, halVersion, deviceVersion, effectiveApiLevel,
                /*out*/&tmp)).isOk()) ①{
            return ret;
        }
        err = client->initialize(mCameraProviderManager);
        ．．．
}

又到了关键的地方，在①处调用了makeClient函数来创建客户端对象：

Status CameraService::makeClient(const sp& cameraService,
        const sp& cameraCb, const String16& packageName, const String8& cameraId,
        int facing, int clientPid, uid_t clientUid, int servicePid, bool legacyMode,
        int halVersion, int deviceVersion, apiLevel effectiveApiLevel,
        /*out*/sp* client) {
    if (halVersion < 0 || halVersion == deviceVersion) {
        // Default path: HAL version is unspecified by caller, create CameraClient
        // based on device version reported by the HAL.
        switch(deviceVersion) {
          case CAMERA_DEVICE_API_VERSION_1_0:
            if (effectiveApiLevel == API_1) {  // Camera1 API route
                sp tmp = static_cast(cameraCb.get());
                *client = new CameraClient(cameraService, tmp, packageName, cameraIdToInt(cameraId),
                        facing, clientPid, clientUid, getpid(), legacyMode);①
            } else { // Camera2 API route
                ALOGW("Camera using old HAL version: %d", deviceVersion);
                return STATUS_ERROR_FMT(ERROR_DEPRECATED_HAL,
                        "Camera device \"%s\" HAL version %d does not support camera2 API",
                        cameraId.string(), deviceVersion);
            }
            break;
          case CAMERA_DEVICE_API_VERSION_3_0:
          case CAMERA_DEVICE_API_VERSION_3_1:
          case CAMERA_DEVICE_API_VERSION_3_2:
          case CAMERA_DEVICE_API_VERSION_3_3:
          case CAMERA_DEVICE_API_VERSION_3_4:
            if (effectiveApiLevel == API_1) { // Camera1 API route
                sp tmp = static_cast(cameraCb.get());
                *client = new Camera2Client(cameraService, tmp, packageName, cameraIdToInt(cameraId),
                        facing, clientPid, clientUid, servicePid, legacyMode);
            } else { // Camera2 API route
                sp tmp =
                        static_cast(cameraCb.get());
                *client = new CameraDeviceClient(cameraService, tmp, packageName, cameraId,
                        facing, clientPid, clientUid, servicePid);
            }
            break;
          default:
            // Should not be reachable
            ALOGE("Unknown camera device HAL version: %d", deviceVersion);
            return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
                    "Camera device \"%s\" has unknown HAL version %d",
                    cameraId.string(), deviceVersion);
        }
    } else {
        // A particular HAL version is requested by caller. Create CameraClient
        // based on the requested HAL version.
        if (deviceVersion > CAMERA_DEVICE_API_VERSION_1_0 &&
            halVersion == CAMERA_DEVICE_API_VERSION_1_0) {
            // Only support higher HAL version device opened as HAL1.0 device.
            sp tmp = static_cast(cameraCb.get());
            *client = new CameraClient(cameraService, tmp, packageName, cameraIdToInt(cameraId),
                    facing, clientPid, clientUid, servicePid, legacyMode);
        } else {
            // Other combinations (e.g. HAL3.x open as HAL2.x) are not supported yet.
            ALOGE("Invalid camera HAL version %x: HAL %x device can only be"
                    " opened as HAL %x device", halVersion, deviceVersion,
                    CAMERA_DEVICE_API_VERSION_1_0);
            return STATUS_ERROR_FMT(ERROR_ILLEGAL_ARGUMENT,
                    "Camera device \"%s\" (HAL version %d) cannot be opened as HAL version %d",
                    cameraId.string(), deviceVersion, halVersion);
        }
    }
    return Status::ok();
}

在makeClient函数里面会通过对HAL版本的判断来创建不同的Client对象，因为本篇文章分析的是HAL1,所以走的是①处代码：

CameraClient.cpp

status_t CameraClient::initialize(sp manager) {
    ...
    mHardware = new CameraHardwareInterface(camera_device_name);
    res = mHardware->initialize(manager);
     ...
    mHardware->setCallbacks(notifyCallback,
            dataCallback,
            dataCallbackTimestamp,
            handleCallbackTimestampBatch,
            (void *)(uintptr_t)mCameraId);
     ...
}

可以看到在CameraClient.cpp的initialize函数中,创建了CameraHardwareInterface类型的对象,然后执行了initialize函数：

HAL层

CameraHardwareInterface.cpp

status_t CameraHardwareInterface::initialize(sp manager) {
ALOGI(“Opening camera %s”, mName.string());

status_t ret = manager->openSession(mName.string(), this, &mHidlDevice);
if (ret != OK) {
    ALOGE("%s: openSession failed! %s (%d)", __FUNCTION__, strerror(-ret), ret);
}
return ret;

}
通过 manager.openSession调用HAL中的库,我们的指令就能传递到 Linux Kernel,从而下达到具体的设备上.（与具体的驱动相关,暂时不去分析）最后初始化预览窗口.HAL层暂时还没有去深究，待研究后再总结一下．

盗用一张图总结一下

参考链接：
https://blog.csdn.net/qq_16775897/article/details/77509476
https://blog.csdn.net/qq_16775897/article/details/77450421