Camera操作过程中最重要的四个步骤：
- CameraManager-->openCamera ---> 打开相机
- CameraDeviceImpl-->createCaptureSession ---> 创建捕获会话
- CameraCaptureSession-->setRepeatingRequest ---> 设置预览界面
- CameraDeviceImpl-->capture ---> 开始捕获图片
之前我们介绍了openCamera流程和createCaptureSession流程，如下：
《Android Camera原理之openCamera模块(一)》
《Android Camera原理之openCamera模块(二)》
《Android Camera原理之createCaptureSession模块》
至此，Camera 会话已经创建成功，接下来我们可以开始预览了，预览回调onCaptureCompleted之后就可以拍照(回调到onCaptureCompleted，说明capture 完整frame数据已经返回了，可以捕捉其中的数据了。)，由于预览和拍照的很多流程很相似，拍照只是预览过程中的一个节点，所以我们把预览和拍照放在一文里讲解。
1.预览
预览发起的函数就是CameraCaptureSession-->setRepeatingRequest，本文我们就谈一下Camera 是如何发起预览操作的。CameraCaptureSession-->setRepeatingRequest是createCaptureSession(List中输出流配置成功之后执行CameraCaptureSession.StateCallback.onConfigured(@NonNull CameraCaptureSession session)函数中执行的。
mCameraDevice.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
new CameraCaptureSession.StateCallback() {
@Override
public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
// The camera is already closed
if (null == mCameraDevice) {
return;
}
// When the session is ready, we start displaying the preview.
mCaptureSession = cameraCaptureSession;
try {
// Auto focus should be continuous for camera preview.
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
// Flash is automatically enabled when necessary.
setAutoFlash(mPreviewRequestBuilder);
// Finally, we start displaying the camera preview.
mPreviewRequest = mPreviewRequestBuilder.build();
mCaptureSession.setRepeatingRequest(mPreviewRequest,
mCaptureCallback, mBackgroundHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
@Override
public void onConfigureFailed(
@NonNull CameraCaptureSession cameraCaptureSession) {
showToast("Failed");
}
}, null
);
最终执行了mCaptureSession.setRepeatingRequest(mPreviewRequest, mCaptureCallback, mBackgroundHandler);
来执行camera preview操作。像对焦等操作就可以在这个onConfigured回调中完成。
onConfigured回调表示当前的配置流已经完成，相机已经显示出来了，可以预览了。onConfigureFailed配置流失败，相机黑屏。
public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
Handler handler) throws CameraAccessException {
checkRepeatingRequest(request);
synchronized (mDeviceImpl.mInterfaceLock) {
checkNotClosed();
handler = checkHandler(handler, callback);
return addPendingSequence(mDeviceImpl.setRepeatingRequest(request,
createCaptureCallbackProxy(handler, callback), mDeviceExecutor));
}
}
- 第一个参数CaptureRequest 标识当前capture 请求的属性，是请求一个camera还是多个camera，是否复用之前的请求等等。
- 第二个参数CaptureCallback 是捕捉回调，这是开发者直接接触的回调。
public interface CaptureCallback {
public static final int NO_FRAMES_CAPTURED = -1;
public void onCaptureStarted(CameraDevice camera,
CaptureRequest request, long timestamp, long frameNumber);
public void onCapturePartial(CameraDevice camera,
CaptureRequest request, CaptureResult result);
public void onCaptureProgressed(CameraDevice camera,
CaptureRequest request, CaptureResult partialResult);
public void onCaptureCompleted(CameraDevice camera,
CaptureRequest request, TotalCaptureResult result);
public void onCaptureFailed(CameraDevice camera,
CaptureRequest request, CaptureFailure failure);
public void onCaptureSequenceCompleted(CameraDevice camera,
int sequenceId, long frameNumber);
public void onCaptureSequenceAborted(CameraDevice camera,
int sequenceId);
public void onCaptureBufferLost(CameraDevice camera,
CaptureRequest request, Surface target, long frameNumber);
}
这需要开发者自己实现，这些回调是如何调用到上层的，请看《Android Camera原理之CameraDeviceCallbacks回调模块》，都是通过CameraDeviceCallbacks回调调上来的。
下面我们从camera 调用原理的角度分析一下mCaptureSession.setRepeatingRequest
--->CameraDeviceImpl.setRepeatingRequest
--->CameraDeviceImpl.submitCaptureRequest
其中CameraDeviceImpl.setRepeatingRequest中第3个参数传入的是true。之所以这个强调一点，因为接下来执行CameraDeviceImpl.capture的时候也会执行setRepeatingRequest，这里第3个参数传入的就是false。第3个参数boolean repeating如果为true，表示当前捕获的是一个过程，camera frame不断在填充；如果为false，表示当前捕获的是一个瞬间，就是拍照。
public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
Executor executor) throws CameraAccessException {
List requestList = new ArrayList();
requestList.add(request);
return submitCaptureRequest(requestList, callback, executor, /*streaming*/true);
}
private int submitCaptureRequest(List requestList, CaptureCallback callback,
Executor executor, boolean repeating) {
//......
}
CameraDeviceImpl.submitCaptureRequest核心工作就是3步：
- 1.验证当前CaptureRequest列表中的request是否合理：核心就是验证与request绑定的Surface是否存在。
- 2.向底层发送请求信息。
- 3.将底层返回的请求信息和传入的CaptureCallback 绑定，以便后续正确回调。
而这三步中，第二步却是核心工作。
1.1 向底层发送captureRequest请求
SubmitInfo requestInfo;
CaptureRequest[] requestArray = requestList.toArray(new CaptureRequest[requestList.size()]);
// Convert Surface to streamIdx and surfaceIdx
for (CaptureRequest request : requestArray) {
request.convertSurfaceToStreamId(mConfiguredOutputs);
}
requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);
if (DEBUG) {
Log.v(TAG, "last frame number " + requestInfo.getLastFrameNumber());
}
for (CaptureRequest request : requestArray) {
request.recoverStreamIdToSurface();
}
- 执行
request.convertSurfaceToStreamId(mConfiguredOutputs);将本地已经缓存的surface和stream记录在内存中，并binder传输到camera service层中，防止camera service端重复请求。requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);这儿直接调用到camera service端。这儿需要重点讲解一下的。request.recoverStreamIdToSurface();回调成功，清除之前在内存中的数据。
CameraDeviceClient::submitRequest
--->CameraDeviceClient::submitRequestList
这个函数代码很多，前面很多执行都是在复用检索之前的缓存是否可用，我们关注一下核心的执行：预览的情况下传入的streaming是true，执行上面；如果是拍照的话，那就执行下面的else。err = mDevice->setStreamingRequestList(metadataRequestList, surfaceMapList, &(submitInfo->mLastFrameNumber));
传入的submitInfo就是要返回上层的回调参数，如果是预览状态，需要不断更新当前的的frame数据，所以每次更新最新的frame number。
if (streaming) {
err = mDevice->setStreamingRequestList(metadataRequestList, surfaceMapList,
&(submitInfo->mLastFrameNumber));
if (err != OK) {
String8 msg = String8::format(
"Camera %s: Got error %s (%d) after trying to set streaming request",
mCameraIdStr.string(), strerror(-err), err);
ALOGE("%s: %s", __FUNCTION__, msg.string());
res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
msg.string());
} else {
Mutex::Autolock idLock(mStreamingRequestIdLock);
mStreamingRequestId = submitInfo->mRequestId;
}
} else {
err = mDevice->captureList(metadataRequestList, surfaceMapList,
&(submitInfo->mLastFrameNumber));
if (err != OK) {
String8 msg = String8::format(
"Camera %s: Got error %s (%d) after trying to submit capture request",
mCameraIdStr.string(), strerror(-err), err);
ALOGE("%s: %s", __FUNCTION__, msg.string());
res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
msg.string());
}
ALOGV("%s: requestId = %d ", __FUNCTION__, submitInfo->mRequestId);
}
Camera3Device::setStreamingRequestList
--->Camera3Device::submitRequestsHelper
status_t Camera3Device::submitRequestsHelper(
const List &requests,
const std::list &surfaceMaps,
bool repeating,
/*out*/
int64_t *lastFrameNumber) {
ATRACE_CALL();
Mutex::Autolock il(mInterfaceLock);
Mutex::Autolock l(mLock);
status_t res = checkStatusOkToCaptureLocked();
if (res != OK) {
// error logged by previous call
return res;
}
RequestList requestList;
res = convertMetadataListToRequestListLocked(requests, surfaceMaps,
repeating, /*out*/&requestList);
if (res != OK) {
// error logged by previous call
return res;
}
if (repeating) {
res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
} else {
res = mRequestThread->queueRequestList(requestList, lastFrameNumber);
}
//......
return res;
}
预览的时候会执行mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
拍照的时候执行mRequestThread->queueRequestList(requestList, lastFrameNumber);
mRequestThread->setRepeatingRequests
status_t Camera3Device::RequestThread::setRepeatingRequests(
const RequestList &requests,
/*out*/
int64_t *lastFrameNumber) {
ATRACE_CALL();
Mutex::Autolock l(mRequestLock);
if (lastFrameNumber != NULL) {
*lastFrameNumber = mRepeatingLastFrameNumber;
}
mRepeatingRequests.clear();
mRepeatingRequests.insert(mRepeatingRequests.begin(),
requests.begin(), requests.end());
unpauseForNewRequests();
mRepeatingLastFrameNumber = hardware::camera2::ICameraDeviceUser::NO_IN_FLIGHT_REPEATING_FRAMES;
return OK;
}
将当前提交的CaptureRequest请求放入之前的预览请求队列中，告知HAL层有新的request请求，HAL层连接请求开始工作，源源不断地输出信息到上层。这儿是跑在Camera3Device中定义的RequestThread线程中，可以保证在预览的时候不断地捕获信息流，camera就不断处于预览的状态了。
1.2 将返回请求信息和 CaptureCallback 绑定
if (callback != null) {
mCaptureCallbackMap.put(requestInfo.getRequestId(),
new CaptureCallbackHolder(
callback, requestList, executor, repeating, mNextSessionId - 1));
} else {
if (DEBUG) {
Log.d(TAG, "Listen for request " + requestInfo.getRequestId() + " is null");
}
}
/** map request IDs to callback/request data */
private final SparseArray mCaptureCallbackMap =
new SparseArray();
1.向底层发送captureRequest请求--->回调的requestIinfo表示当前capture request的结果，将requestInfo.getRequestId()与CaptureCallbackHolder绑定，因为Camera 2架构支持发送多次CaptureRequest请求，如果不使用这种绑定机制，后续的回调会造成严重的错乱，甚至回调不上来，那么开发者无法继续使用了。
我们看看使用这些回调的地方的代码：
《Android Camera原理之CameraDeviceCallbacks回调模块》已经说明了CameraDeviceCallbacks.aidl才是camera service进程与用户进程通信的回调，到这个回调里面，再取出CaptureRequest绑定的CaptureCallback回调，调用到CaptureCallback回调函数，这样开发者可以直接使用。
下面看看CameraDeviceCallbacks的onCaptureStarted回调---->
public void onCaptureStarted(final CaptureResultExtras resultExtras, final long timestamp) {
int requestId = resultExtras.getRequestId();
final long frameNumber = resultExtras.getFrameNumber();
if (DEBUG) {
Log.d(TAG, "Capture started for id " + requestId + " frame number " + frameNumber);
}
final CaptureCallbackHolder holder;
synchronized(mInterfaceLock) {
if (mRemoteDevice == null) return; // Camera already closed
// Get the callback for this frame ID, if there is one
holder = CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);
if (holder == null) {
return;
}
if (isClosed()) return;
// Dispatch capture start notice
final long ident = Binder.clearCallingIdentity();
try {
holder.getExecutor().execute(
new Runnable() {
@Override
public void run() {
if (!CameraDeviceImpl.this.isClosed()) {
final int subsequenceId = resultExtras.getSubsequenceId();
final CaptureRequest request = holder.getRequest(subsequenceId);
if (holder.hasBatchedOutputs()) {
// Send derived onCaptureStarted for requests within the
// batch
final Range fpsRange =
request.get(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE);
for (int i = 0; i < holder.getRequestCount(); i++) {
holder.getCallback().onCaptureStarted(
CameraDeviceImpl.this,
holder.getRequest(i),
timestamp - (subsequenceId - i) *
NANO_PER_SECOND/fpsRange.getUpper(),
frameNumber - (subsequenceId - i));
}
} else {
holder.getCallback().onCaptureStarted(
CameraDeviceImpl.this,
holder.getRequest(resultExtras.getSubsequenceId()),
timestamp, frameNumber);
}
}
}
});
} finally {
Binder.restoreCallingIdentity(ident);
}
}
}
holder = CameraDeviceImpl.this.mCaptureCallbackMap.get(requestId);然后直接调用
holder.getCallback().onCaptureStarted(
CameraDeviceImpl.this,
holder.getRequest(i),
timestamp - (subsequenceId - i) *
NANO_PER_SECOND/fpsRange.getUpper(),
frameNumber - (subsequenceId - i));
简单明了，脉络清楚。
2.拍照
开发者如果想要拍照的话，直接调用mCaptureSession.capture(mPreviewRequestBuilder.build(), mCaptureCallback, mBackgroundHandler);
拍照的调用流程和预览很相似，只是在调用函数中个传入的参数不同。
public int capture(CaptureRequest request, CaptureCallback callback, Executor executor)
throws CameraAccessException {
if (DEBUG) {
Log.d(TAG, "calling capture");
}
List requestList = new ArrayList();
requestList.add(request);
return submitCaptureRequest(requestList, callback, executor, /*streaming*/false);
}
拍照的时候也是调用submitCaptureRequest，只不过第3个参数传入的是false，表示不用循环获取HAL调用上来的帧数据，只获取瞬间的帧数据就可以。
拍照和预览调用的区分在：CameraDeviceClient::submitRequestList
if (streaming) {
//......
} else {
err = mDevice->captureList(metadataRequestList, surfaceMapList,
&(submitInfo->mLastFrameNumber));
if (err != OK) {
String8 msg = String8::format(
"Camera %s: Got error %s (%d) after trying to submit capture request",
mCameraIdStr.string(), strerror(-err), err);
ALOGE("%s: %s", __FUNCTION__, msg.string());
res = STATUS_ERROR(CameraService::ERROR_INVALID_OPERATION,
msg.string());
}
ALOGV("%s: requestId = %d ", __FUNCTION__, submitInfo->mRequestId);
}
接下里调用到mDevice->captureList
--->Camera3Device::submitRequestsHelper
status_t Camera3Device::submitRequestsHelper(
const List &requests,
const std::list &surfaceMaps,
bool repeating,
/*out*/
int64_t *lastFrameNumber) {
//......
RequestList requestList;
//......
if (repeating) {
res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
} else {
res = mRequestThread->queueRequestList(requestList, lastFrameNumber);
}
//......
return res;
}
执行Camera3Device::RequestThread线程中的queueRequestList。
status_t Camera3Device::RequestThread::queueRequestList(
List > &requests,
/*out*/
int64_t *lastFrameNumber) {
ATRACE_CALL();
Mutex::Autolock l(mRequestLock);
for (List >::iterator it = requests.begin(); it != requests.end();
++it) {
mRequestQueue.push_back(*it);
}
if (lastFrameNumber != NULL) {
*lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
ALOGV("%s: requestId %d, mFrameNumber %" PRId32 ", lastFrameNumber %" PRId64 ".",
__FUNCTION__, (*(requests.begin()))->mResultExtras.requestId, mFrameNumber,
*lastFrameNumber);
}
unpauseForNewRequests();
return OK;
}
*lastFrameNumber = mFrameNumber + mRequestQueue.size() - 1;
这里有关键的执行代码，表示当前取最新的capture frame数据。
拍照的时候在什么地方捕捉image？
camera1的时候提供了PictureCallback回调方式来提供实时预览回调，可以在这里获取image数据回调。
camera2没有这个接口，但是提供了ImageReader.OnImageAvailableListener来实现回调。
public interface OnImageAvailableListener {
/**
* Callback that is called when a new image is available from ImageReader.
*
* @param reader the ImageReader the callback is associated with.
* @see ImageReader
* @see Image
*/
void onImageAvailable(ImageReader reader);
}
还记得《Android Camera模块解析之拍照》中提到openCamera之前要设置
mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(),
ImageFormat.JPEG, /*maxImages*/2);
mImageReader.setOnImageAvailableListener(
mOnImageAvailableListener, mBackgroundHandler);
ImageReader中有一个getSurface()函数，这是ImageReader的拍照输出流，我们拍照的时候一般有两个输出流(outputSurface对象)，一个是预览流，还有一个是拍照流。不记得可以参考《Android Camera原理之createCaptureSession模块》，ImageReader设置的拍照流会设置到camera service端。
public Surface getSurface() {
return mSurface;
}
我们看看在什么时候回调这个接口。
ImageReader回调接口.jpg
看上面的调用流程，调用到ImageReader.OnImageAvailableListener->onImageAvailable中，我们获取ImageReader->acquireNextImage可以获取采集的image图片。其实ImageReader中也可以获取预览的流式数据。SurfacePlane 封装了返回的ByteBuffer数据，可供开发者实时获取。
private class SurfacePlane extends android.media.Image.Plane {
private SurfacePlane(int rowStride, int pixelStride, ByteBuffer buffer) {
mRowStride = rowStride;
mPixelStride = pixelStride;
mBuffer = buffer;
/**
* Set the byteBuffer order according to host endianness (native
* order), otherwise, the byteBuffer order defaults to
* ByteOrder.BIG_ENDIAN.
*/
mBuffer.order(ByteOrder.nativeOrder());
}
@Override
public ByteBuffer getBuffer() {
throwISEIfImageIsInvalid();
return mBuffer;
}
final private int mPixelStride;
final private int mRowStride;
private ByteBuffer mBuffer;
}
Note：很多开发者在camera1使用Camera.PreviewCallback的void onPreviewFrame(byte[] data, Camera camera)
可以获取实时数据，但是在camera2中没有这个接口了，虽然camera1的接口方法也能用，camera2替代的接口就是ImageReader.OnImageAvailableListener->onImageAvailable