为什么相机库CameraView预览和拍照的效果不一致 ?

1. 前言

在项目中,我们经常使用到CameraView,遇到好几次预览和拍照实际的效果不一致的情况。

这是为什么呢,为什么使用滤镜的时候,CameraView拍照和预览的效果会不一致 ?

终于在CameraView的源码里找到了答案。

2. takePictureSnapshot

带滤镜拍照的入口是CameraView.takePictureSnapshot()

cameraView.takePictureSnapshot()

cameraView.takePictureSnapshot()会调用到mCameraEngine.takePictureSnapshot()
这个PictureResult.Stub是一个参数封装类,这里重新创建了一个PictureResult.Stub并传入takePictureSnapshot()方法中。
mCameraEngineCameraEngine抽象类,实现类有Camera1EngineCamera2Engine

public void takePictureSnapshot() {
    PictureResult.Stub stub = new PictureResult.Stub();
    mCameraEngine.takePictureSnapshot(stub);
}

我们这里以Camera2为例,可以看到这里对stub参数封装类赋值了一些参数(摄像头ID、图片格式等),并调用了onTakePicture

@Override
public void takePictureSnapshot(final PictureResult.Stub stub) {
    final boolean metering = mPictureSnapshotMetering;
    if (isTakingPicture()) return;
    
    stub.location = mLocation;
    stub.isSnapshot = true;
    stub.facing = mFacing;
    stub.format = PictureFormat.JPEG;
    
    AspectRatio ratio = AspectRatio.of(getPreviewSurfaceSize(Reference.OUTPUT));
    onTakePictureSnapshot(stub, ratio, metering);
}

3. onTakePictureSnapshot

这里给参数封装类stub赋值了一些值,然后初始化Snapshot2PictureRecorder,并调用其take()方法

@Override
protected void onTakePictureSnapshot(@NonNull final PictureResult.Stub stub,
                                     @NonNull final AspectRatio outputRatio,
                                     boolean doMetering) {
    stub.size = getUncroppedSnapshotSize(Reference.OUTPUT);
    stub.rotation = getAngles().offset(Reference.VIEW, Reference.OUTPUT, Axis.ABSOLUTE);
    mPictureRecorder = new Snapshot2PictureRecorder(stub, this, (RendererCameraPreview) mPreview, outputRatio);
    mPictureRecorder.take();
}

4. Snapshot2PictureRecorder.take

Snapshot2PictureRecorder继承自SnapshotGlPictureRecorder,我们这里直接来看SnapshotGlPictureRecorder.take()

public void take() {
    mPreview.addRendererFrameCallback(new RendererFrameCallback() {

        @RendererThread
        public void onRendererTextureCreated(int textureId) {
            SnapshotGlPictureRecorder.this.onRendererTextureCreated(textureId);
        }

        @RendererThread
        @Override
        public void onRendererFilterChanged(@NonNull Filter filter) {
            SnapshotGlPictureRecorder.this.onRendererFilterChanged(filter);
        }

        @RendererThread
        @Override
        public void onRendererFrame(@NonNull SurfaceTexture surfaceTexture,
                                    int rotation, float scaleX, float scaleY) {
            mPreview.removeRendererFrameCallback(this);
            SnapshotGlPictureRecorder.this.onRendererFrame(surfaceTexture,
                                                           rotation, scaleX, scaleY);
        }

    });
}

这里的mPreview实际上是GlCameraPreview,而通过RendererFrameCallback回调,会触发如下几个方法 onRendererTextureCreatedonRendererFilterChangedonRendererFrame

4.1 onRendererFilterChanged

当重新设置Filter的时候会调用这个回调。onRendererFilterChanged里会将filter拷贝一份,赋值给TextureDrawer

protected void onRendererFilterChanged(@NonNull Filter filter) {
    mTextureDrawer.setFilter(filter.copy());
}

4.2 onRendererFrame

OpenGL绘制的时候,会调用onRendererFrameonRendererFrame里会调用takeFrame()

protected void onRendererFrame(final SurfaceTexture surfaceTexture,
                               final int rotation,
                               final float scaleX,
                               final float scaleY) {
    final EGLContext eglContext = EGL14.eglGetCurrentContext();
    takeFrame(surfaceTexture, rotation, scaleX, scaleY, eglContext);
}

5. takeFrame

5.1 创建EGL窗口

首先,会创建EGL窗口,这里创建了一个假的,前台不可见的一个EGL窗口,专门用来保存图片

// 0. EGL window will need an output.
// We create a fake one as explained in javadocs.
final int fakeOutputTextureId = 9999;
SurfaceTexture fakeOutputSurface = new SurfaceTexture(fakeOutputTextureId);
fakeOutputSurface.setDefaultBufferSize(mResult.size.getWidth(), mResult.size.getHeight());

5.2 创建EGL Surface

接着,来创建EglSurface

// 1. Create an EGL surface
final EglCore core = new EglCore(eglContext, EglCore.FLAG_RECORDABLE);
final EglSurface eglSurface = new EglWindowSurface(core, fakeOutputSurface);
eglSurface.makeCurrent();
5.2.1 EglSurface

其中,这个com.otaliastudios.opengl.EglSurface是作者自己创建的,继承自EglNativeSurface,其内部调用了EglCore

public expect abstract class EglSurface internal constructor(eglCore: EglCore, eglSurface: EglSurface) : EglNativeSurface

public open class EglNativeSurface internal constructor(
        internal var eglCore: EglCore,
        internal var eglSurface: EglSurface) {

    public fun getWidth(): Int {
        return if (width < 0) {
            eglCore.querySurface(eglSurface, EGL_WIDTH)
        } else {
            width
        }
    }

    public fun getHeight(): Int {
        return if (height < 0) {
            eglCore.querySurface(eglSurface, EGL_HEIGHT)
        } else {
            height
        }
    }

    public open fun release() {
        eglCore.releaseSurface(eglSurface)
        eglSurface = EGL_NO_SURFACE
        height = -1
        width = -1
    }

    public fun isCurrent(): Boolean {
        return eglCore.isSurfaceCurrent(eglSurface)
    }
}
5.2.2 EglCore

可以看到EglNativeSurface内部其实调用了EglCoreEglCore内部封装了EGL相关的方法。
这里的具体实现我们不需要细看,只需要知道EglSurface是作者自己实现的一个Surface就可以了,内部封装了EGL,可以实现和GlSurfaceView类似的一些功能,在这里使用的EglSurface是专门给拍照准备的。

OpenGL是一个跨平台的操作GPUAPIOpenGL需要本地视窗系统进行交互,就需要一个中间控制层。
EGL就是连接OpenGL ES和本地窗口系统的接口,引入EGL就是为了屏蔽不同平台上的区别。

public expect class EglCore : EglNativeCore

public open class EglNativeCore internal constructor(sharedContext: EglContext = EGL_NO_CONTEXT, flags: Int = 0) {

    //...省略了代码...
}

5.3 修改transform

这里的mTextureDrawerGlTextureDrawerGlTextureDrawer是一个绘制的管理类,无论是GlCameraPreview(预览)还是SnapshotGlPictureRecorder(带滤镜拍照),都是调用GlTextureDrawer.draw()来渲染openGL的。

public class GlTextureDrawer {
	//...省略了不重要的代码...

    private final GlTexture mTexture;
    private float[] mTextureTransform = Egloo.IDENTITY_MATRIX.clone();

    public void draw(final long timestampUs) {
        //...省略了不重要的代码...
        
        if (mProgramHandle == -1) {
            mProgramHandle = GlProgram.create(
                    mFilter.getVertexShader(),
                    mFilter.getFragmentShader());
            mFilter.onCreate(mProgramHandle);
        }

        GLES20.glUseProgram(mProgramHandle);
        mTexture.bind();
        mFilter.draw(timestampUs, mTextureTransform);
        mTexture.unbind();
        GLES20.glUseProgram(0);
    }

    public void release() {
        if (mProgramHandle == -1) return;
        mFilter.onDestroy();
        GLES20.glDeleteProgram(mProgramHandle);
        mProgramHandle = -1;
    }
}

transform ,也就是mTextureTransform,会传到Filter.draw()中,最终会改变OpenGL绘制的坐标矩阵,也就是GLSL中的uMVPMatrix变量。
而这边就是修改transform 的值,从而对图像进行镜像、旋转等操作。

final float[] transform = mTextureDrawer.getTextureTransform();

// 2. Apply preview transformations
surfaceTexture.getTransformMatrix(transform);
float scaleTranslX = (1F - scaleX) / 2F;
float scaleTranslY = (1F - scaleY) / 2F;
Matrix.translateM(transform, 0, scaleTranslX, scaleTranslY, 0);
Matrix.scaleM(transform, 0, scaleX, scaleY, 1);

// 3. Apply rotation and flip
 // If this doesn't work, rotate "rotation" before scaling, like GlCameraPreview does.
 Matrix.translateM(transform, 0, 0.5F, 0.5F, 0); // Go back to 0,0
 Matrix.rotateM(transform, 0, rotation + mResult.rotation, 0, 0, 1); // Rotate to OUTPUT
 Matrix.scaleM(transform, 0, 1, -1, 1); // Vertical flip because we'll use glReadPixels
 Matrix.translateM(transform, 0, -0.5F, -0.5F, 0); // Go back to old position

6. 绘制并保存

这里就是带滤镜拍照部分,核心中的核心代码了。
这里主要分为两步

  • mTextureDrawer.draw : 绘制滤镜
  • eglSurface.toByteArray : 将画面保存为JPEG格式的Byte数组
// 5. Draw and save
long timestampUs = surfaceTexture.getTimestamp() / 1000L;
LOG.i("takeFrame:", "timestampUs:", timestampUs);
mTextureDrawer.draw(timestampUs);
if (mHasOverlay) mOverlayDrawer.render(timestampUs);
mResult.data = eglSurface.toByteArray(Bitmap.CompressFormat.JPEG);

6.1 mTextureDrawer.draw

绘制滤镜

public void draw(final long timestampUs) {
    if (mPendingFilter != null) {
        release();
        mFilter = mPendingFilter;
        mPendingFilter = null;

    }

    if (mProgramHandle == -1) {
        mProgramHandle = GlProgram.create(
                mFilter.getVertexShader(),
                mFilter.getFragmentShader());
        mFilter.onCreate(mProgramHandle);
        Egloo.checkGlError("program creation");
    }

    GLES20.glUseProgram(mProgramHandle);
    Egloo.checkGlError("glUseProgram(handle)");
    mTexture.bind();
    mFilter.draw(timestampUs, mTextureTransform);
    mTexture.unbind();
    GLES20.glUseProgram(0);
    Egloo.checkGlError("glUseProgram(0)");
}

6.2 eglSurface.toByteArray

将画面保存为JPEG格式的Byte数组

public fun toByteArray(format: Bitmap.CompressFormat = Bitmap.CompressFormat.PNG): ByteArray {
    val stream = ByteArrayOutputStream()
        stream.use {
        toOutputStream(it, format)
            return it.toByteArray()
    }
}
public fun toOutputStream(stream: OutputStream, format: Bitmap.CompressFormat) {
    val width = getWidth()
    val height = getHeight()
    val buf = ByteBuffer.allocateDirect(width * height * 4)
    buf.order(ByteOrder.LITTLE_ENDIAN)
    GLES20.glReadPixels(0, 0, width, height, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, buf)

    buf.rewind()
    val bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
    bitmap.copyPixelsFromBuffer(buf)
    bitmap.compress(format, 90, stream)
    bitmap.recycle()
}

7. 分发回调

最后,调用dispatchResult分发回调

protected void dispatchResult() {
    if (mListener != null) {
        mListener.onPictureResult(mResult, mError);
        mListener = null;
        mResult = null;
    }
}

最终会回调CameraViewmListeners列表中的onPictureTaken()方法

mListeners什么时候被添加呢 ? CameraView中有一个addCameraListener方法,专门用来添加回调。

public void addCameraListener(CameraListener cameraListener) {
    mListeners.add(cameraListener);
}

8. 设置回调

所以我们只要添加了这个回调,并实现onPictureTaken方法,就可以在onPictureTaken()中获取到拍照后的图像信息了。

binding.cameraView.addCameraListener(object : CameraListener() {
    override fun onPictureTaken(result: PictureResult) {
        super.onPictureTaken(result)
        //拍照回调
        val bitmap = BitmapFactory.decodeByteArray(result.data, 0, result.data.size)
        bitmap?.also {
            Toast.makeText(this@Test2Activity, "拍照成功", Toast.LENGTH_SHORT).show()
            //将Bitmap设置到ImageView上
            binding.img.setImageBitmap(it)
            
            val file = getNewImageFile()
            //保存图片到指定目录
            ImageUtils.save(it, file, Bitmap.CompressFormat.JPEG)
        }
    }
})

9. 回到开始的问题

为什么CameraView预览和拍照的效果不一致 ?

EglSurface是作者自己实现的一个Surface,可以实现和GlSurfaceView类似的一些功能。

CameraView中,GlSurfaceView是专门用来预览,而作者自己实现的EglSurface是用来拍照时候存储图像的。

这样做的好处在于拍照的时候,预览界面(GLSurfaceView)不会出现卡顿的现象,但是坏处也显而易见,就是可能会出现预览效果和拍照的实际效果不一致的情况。

10. 其他

10.1 CameraView源码解析系列

Android 相机库CameraView源码解析 (一) : 预览-CSDN博客
Android 相机库CameraView源码解析 (二) : 拍照-CSDN博客
Android 相机库CameraView源码解析 (三) : 滤镜相关类说明-CSDN博客
Android 相机库CameraView源码解析 (四) : 带滤镜拍照-CSDN博客
Android 相机库CameraView源码解析 (五) : 保存滤镜效果-CSDN博客

你可能感兴趣的:(Android,CameraView,滤镜,拍照,预览,效果不一致,filter)