MTK Camera学习第四篇(拍照流程)

本篇仅学习从应用层到framework的过程,jni以下部分暂不讨论。因为一个优秀的相机应用,核心永远是它的图像处理部分(即Hal层中的3A算法部分),而MTK相关内容未开源。我们从按下拍照键开始做时序图如下:


MTK Camera学习第四篇(拍照流程)_第1张图片
camera-takephoto.png

上图可以分成三个部分,第一部分就是接口的回调与实现,第二部分是不同模块execute执行指令部分,第三个部分是数据保存生成jpeg文件。我们这里说一下时序图上未记录的一些东西:
PhotoActor.java

    @Override
    public void onShutterButtonClick(ShutterButton button) {
        int cameraState = mCameraActivity.getCameraState();
        ViewState currentViewState = mICameraAppUi.getViewState();
        Log.i(TAG, "[onShutterButtonClick] cameraState = " + cameraState
                + ", currentViewState = " + currentViewState);

        if (ViewState.VIEW_STATE_LOMOEFFECT_SETTING == currentViewState
                || ViewState.VIEW_STATE_CONTINUOUS_CAPTURE == currentViewState
                || ViewState.VIEW_STATE_CAMERA_CLOSED == currentViewState) {
            return;
        }

        if (mICameraAppUi.updateRemainStorage() > 0) {
            if (!(CameraActivity.STATE_SWITCHING_CAMERA == cameraState || CameraActivity.STATE_PREVIEW_STOPPED == cameraState)) {
                if (mSelfTimerManager.startSelfTimer()) {
                    Log.i(TAG, "[onShutterButtonClick] start self timer");
                    mModuleManager.onSelfTimerState(true);
                    mICameraAppUi.setSwipeEnabled(false);
                    mICameraAppUi.setViewState(ViewState.VIEW_STATE_CAPTURE);
                    mIsSelftimerCounting = true;
                    return;
                } else {
                    mIsSelftimerCounting = false;
                }
                
                mModuleManager.onPhotoShutterButtonClick();
            }
        } else {
            Log.i(TAG, "remain storage is less than 0");
            mICameraAppUi.showRemaining();
        }
    }

这里面有两个函数未记入时序图,一个updateRemainStorage(),其最终实现位于RemainingManager用来计算当前的剩余空间,空间不足自然无法拍照保存,另一个是mSelfTimerManager.startSelfTimer()用来做倒计时拍照的功能。

ModuleManager.java

    public boolean onPhotoShutterButtonClick() {
        mAdditionManager.execute(ActionType.ACTION_PHOTO_SHUTTER_BUTTON_CLICK, false);
        return mICameraMode.execute(ActionType.ACTION_PHOTO_SHUTTER_BUTTON_CLICK);
    }

在这个地方开始执行execute,mAdditionManager是什么呢?我们进入这个类,看一下它的execute方法:

   public boolean execute(AdditionActionType type, Object... arg) {
        Log.i(TAG, "[execute],addition action type = " + type);
        boolean result = false;
        for (ICameraAddition addition : mModeAddition) {
            result = addition.execute(type, arg) || result;
        }
        return result;
    }

mModeAddition表示当前的拍摄模式,其值定义如下:

    public void setCurrentMode(CameraModeType type) {
        Log.i(TAG, "[setCurrentMode]type = " + type);
        switch (type) {
        case EXT_MODE_PHOTO:
            mModeAddition = mPhotoAddtion;
            break;

        case EXT_MODE_VIDEO:
            mModeAddition = mVideoAddtion;
            break;

        case EXT_MODE_PHOTO_PIP:
            mModeAddition = mPipPhotoAddition;
            break;

        case EXT_MODE_VIDEO_PIP:
            mModeAddition = mPipVideoAddition;
            break;

        case EXT_MODE_FACE_BEAUTY:
            mModeAddition = mFaceBeautyAddition;
            break;

        case EXT_MODE_STEREO_CAMERA:
            mModeAddition = mRefocusAddition;
            break;
        default:
            mModeAddition = mDummyAddtion;
            break;
        }
    }

我们从相关各个类的关系来看下:


MTK Camera学习第四篇(拍照流程)_第2张图片
camera-addition.png

CameraAddition的子类除了图中的还有其它,这里只画出了在AdditionManager初始化时用到的子类。整个addition包是由mtk新加的code,从ICameraAddition该接口的定义来看,大概是将不同的ACTION_TYPE传递到不同的MODE下进行Camera的操作。

回到前面mICameraMode.execute(),根据不同的mode进入相关类,我们这里肯定是执行PhotoMode的execute方法。顺序执行到如下方法:

    private void onShutterButtonClick() {
        boolean isEnoughSpace = mIFileSaver.isEnoughSpace();
        Log.i(TAG, "[onShutterButtonClick]isEnoughSpace = " + isEnoughSpace + ",mCameraClosed = "
                + mCameraClosed + ",mCurrentState = " + getModeState());
        // Do not take the picture if there is not enough storage or camera is not available.
        if (!isEnoughSpace || isCameraNotAvailable()) {
            Log.w(TAG, "[onShutterButtonClick]return.");
            return;
        }
        Log.i(TAG,
                "[CMCC Performance test][Camera][Camera] camera capture start ["
                        + System.currentTimeMillis() + "]");
        if (mIFocusManager != null) {
            mIFocusManager.focusAndCapture();
        }
    }

这里FileSaverImpl会判断一次剩余空间的大小,在前面第一次判断剩余空间时,将avaliableSpace保存到Storage,而在这里会将该值取出再次判断。而当相机处于不可用状态时也将放弃拍照而直接返回。

    private boolean isCameraNotAvailable() {
        ModeState modeState = getModeState();
        Log.d(TAG, "isCameraNotAvailable modeState " + modeState);
        return (ModeState.STATE_CAPTURING == modeState || ModeState.STATE_SAVING == modeState ||
                ModeState.STATE_CLOSED == modeState) ? true : false;
    }

准确的来说,应该不是相机不可用,而是当相机处于CAPTURING、SAVING、CLOSED这三种状态时暂时看作相机不可用。如果不是这三种状态,则开始聚焦拍照。然后,接口回调到PhotoMode中的capture()方法:

    @Override
    public boolean capture() {
        Log.i(TAG, "[capture]...");

        long start = System.currentTimeMillis();
        mCaptureStartTime = System.currentTimeMillis();
        mPostViewPictureCallbackTime = 0;
        mJpegImageData = null;
        mIFileSaver.init(FILE_TYPE.JPEG, 0, null, -1);
        mICameraAppUi.setSwipeEnabled(false);
        mICameraAppUi.showRemaining();
        mCameraCategory.takePicture();
        setModeState(ModeState.STATE_CAPTURING);

        Log.d(TAG, "[capture] Capture time = " + (System.currentTimeMillis() - start));
        return true;
    }

这里各种工作准备就绪后初始化FileSaver相关准备进行文件保存,然后更新UI显示当前的剩余空间,然后调用了内部类的方法再通过代理调用到了Camera。

   protected class CameraCategory {

        public void takePicture() {
            if (!mAdditionManager.execute(AdditionActionType.ACTION_TAKEN_PICTURE)) {
                mICameraDevice.takePicture(mShutterCallback, mRawPictureCallback,
                        null, mJpegPictureCallback);
                mICameraAppUi.setViewState(ViewState.VIEW_STATE_CAPTURE);
            }
        }
    }

我们的文件保存在哪进行的呢,看下这个mJpegPictureCallback:

 private final PictureCallback mJpegPictureCallback = new PictureCallback() {
        @Override
        public void onPictureTaken(byte[] jpegData, Camera camera) {
            Log.i(TAG, "[mJpegPictureCallback]onPictureTaken");
            ...
            // Calculate the width and the height of the jpeg.
            if (!mIModuleCtrl.isImageCaptureIntent()) {
                mIFileSaver.savePhotoFile(jpegData, null, mCaptureStartTime,
                        mIModuleCtrl.getLocation(), 0, null);
            } else {
                mJpegImageData = jpegData;
                if (!mIModuleCtrl.isQuickCapture()) {
                    mICameraAppUi.showReview(null, null);
                    mICameraAppUi.switchShutterType(ShutterButtonType.SHUTTER_TYPE_OK_CANCEL);
                } else {
                    doAttach();
                }
            }
           ...
        }
    };

SaveRequest将保存照片的各种信息:

    @Override
    public boolean savePhotoFile(byte[] photoData, String fileName, long date,
            Location location, int tag, OnFileSavedListener listener) {
        Log.i(TAG, "[savePhotoFile]title =" + fileName);
        if (null == mSaveRequest || null == photoData) {
            Log.w(TAG, "[savePhotoFile]fail,mSaveRequest = " + mSaveRequest);
            return false;
        }

        mListener = listener;
        if (mSaveRequest.getDataSize() > 0) {
            Log.i(TAG, "[savePhotoFile]Current SaveRequest is used, copy new one!");
            mSaveRequest = mFileSaver.copyPhotoRequest(mSaveRequest);
        }
        mSaveRequest.setData(photoData);
        mSaveRequest.setFileName(fileName);
        mSaveRequest.setTag(tag);
        mSaveRequest.updateDataTaken(date);
        mSaveRequest.setLocation(location);
        mSaveRequest.setListener(mFileSaverListener);
        mSaveRequest.addRequest();

        return true;
    }

SaveRequest是一个接口,其实现在FileSaver的内部类RequestOperator,而该内部类同时又是一个抽象类,其子类同样是位于FileSaver的内部类VideoOperator、PanoOperator、PhotoOperator,因此这里的addRequet方法最终由PhotoOperator实现,它又将调用外部类FileSaver的addSaveRequest()方法,而这个请求在这里将交给FileSaverService来处理:

    // run in main thread
    public void addSaveRequest(SaveRequest request) {
        Log.i(TAG, "[addSaveRequest]...begin,the queue number is = " + mQueue.size()
                + "mContinuousSaveTask:" + mContinuousSaveTask);
        synchronized (mQueue) {
            mQueue.add(request);
        }
        if (mContinuousSaveTask == null) {
            mContinuousSaveTask = new SaveTask();
            mTaskNumber++;
            mContinuousSaveTask.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR);
            Log.i(TAG, "[addSaveRequest]execute continuous AsyncTask.");
        }
        Log.i(TAG, "[addSaveRequest]...end,the queue number is = " + mQueue.size());

    }

因为需要写文件,这里开启了一个AsyncTask来处理耗时操作:

        @Override
        protected Void doInBackground(Void... v) {
            Log.i(TAG, "[SaveTask]doInBackground...,queue is empty = " + mQueue.isEmpty());
            FileSaverListener lastFileSaverListener = null;
            while (!mQueue.isEmpty()) {
                r = mQueue.get(0);
                //different cameraActivity use different listener
                // notify old listener save done info
                if (lastFileSaverListener != null
                        && r.getFileSaverListener() != lastFileSaverListener) {
                    r.getFileSaverListener().onSaveDone();
                }

                if (Storage.isStorageReady()) {
                    r.saveRequest();
                }
                r.notifyListener();
                // LinkedList is not saved list, so mQueue should be sync in
                // multi thread
                synchronized (mQueue) {
                    mQueue.remove(0);
                }
                synchronized (mListnerObject) {
                    r.getFileSaverListener().onFileSaved(r);
                }
                lastFileSaverListener = r.getFileSaverListener();
            }
            mContinuousSaveTask = null;
            mTaskNumber--;
            synchronized (mListnerObject) {
                r.getFileSaverListener().onSaveDone();
            }
            Log.i(TAG, "[SaveTask]doInBackground...,end ");
            return null;
        }

通知相关Listener保存成功。SD卡准备好的话,具体的写文件操作重新交给SaveRequest(由java特性确定最后由PhotoOperator实现)


        @Override
        public synchronized void saveRequest() {
            if (mData == null) {
                Log.w(TAG, "[saveRequest]mData is null,return!");
                return;
            }
            int orientation = Exif.getOrientation(mData);
            // M: ConShots
            mGroupId = Exif.getGroupId(mData);
            mGroupIndex = Exif.getGroupIndex(mData);
            mFocusValueHigh = Exif.getFocusValueHigh(mData);
            mFocusValueLow = Exif.getFocusValueLow(mData);
            mOrientation = orientation;
            mDataSize = mData.length;

            if (null != mFileName) {
                mTitle = mFileName.substring(0, mFileName.indexOf('.'));
            } else {
                mTitle = createName(mFileType, mDateTaken, mGroupIndex);
                mFileName = Storage.generateFileName(mTitle, mTempPictureType);
                Log.i(TAG, "[saveRequest]PhotoOperator,mFileName = " + mFileName);
            }
            mFilePath = Storage.generateFilepath(mFileName);
            mTempFilePath = mFilePath + TEMP_SUFFIX;
            saveImageToSDCard(mTempFilePath, mFilePath, mData);
            // camera decouple
            mMimeType = Storage.generateMimetype(mTitle, mTempPictureType);
            checkDataProperty();
            saveImageToDatabase(this);
        }
        private void saveImageToSDCard(String tempFilePath, String filePath, byte[] data) {
            FileOutputStream out = null;
            try {
                // Write to a temporary file and rename it to the final name.
                // This
                // avoids other apps reading incomplete data.
                Log.d(TAG, "[saveImageToSDCard]begin add the data to SD Card");
                out = new FileOutputStream(tempFilePath);
                out.write(data);
                out.close();
                new File(tempFilePath).renameTo(new File(filePath));
            } catch (IOException e) {
                Log.e(TAG, "[saveImageToSDCard]Failed to write image,ex:", e);
            } finally {
                if (out != null) {
                    try {
                        out.close();
                    } catch (IOException e) {
                        Log.e(TAG, "[saveImageToSDCard]IOException:", e);
                    }
                }
            }
            Log.i(TAG, "[saveImageToSDCard]end of add the data to SD Card");
        }

我们看到最后的文件生成就是一个new File的过程,这说明我们在PictureCallback里面拿到的数据一定是已经过3A算法处理后的数据,而不是Camera设备得到的原始数据。最后我们看下FileSaver相关类的关系图:


MTK Camera学习第四篇(拍照流程)_第3张图片
camera-filesaver.png

为方便看图,这里没有加入VideoOperator和PanoOperator这两个内部类。我们在PictureCallback中将数据保存为文件的时候,直接操作的是IFileSaver这个接口,其实现是FileSaverImpl,然后该类中有两个成员变量mSaveRequest和mVideoSaveRequest(接口SaveRequest的对象),执行它们的addRequest()方法也就是执行FileSaver的内部类RequestOperator的想着方法,而由于其抽象类型的原因最终又将来到其子类PhotoOperator的相关方法,与我们上面的时序图一致。

你可能感兴趣的:(MTK Camera学习第四篇(拍照流程))