Android 实时滤镜 Demo（GPUImage + Camera2 实现）

一、应用截图

二、前言

GPUImage 是一个开源的图像渲染的库，使用它可以轻松实现很多滤镜效果，也可以很轻松的定义和实现自己特有的滤镜效果。

地址：https://github.com/cats-oss/android-gpuimage

三、依赖工程

要想使用 GPUImage，使用 Android Studio 只需要在 build.gradle 里面添加相关的依赖即可。

implementation 'jp.co.cyberagent.android:gpuimage:2.0.3'

关于 GPUImage 的一些类介绍，在后面再说，先熟悉使用吧。

四、工程代码

首先工程是仿照 GPUImage 中的 Sample 写的，很多代码都是借鉴的，改了改 UI ，优化了一些使用。

1. 相机布局

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
              android:layout_width="match_parent"
              android:layout_height="match_parent"
              android:background="@android:color/black"
              android:orientation="vertical">

    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:orientation="horizontal">

        <ImageView
            android:id="@+id/close_iv"
            android:layout_width="40dp"
            android:layout_height="40dp"
            android:layout_marginLeft="10dp"
            android:padding="8dp"
            android:src="@mipmap/ic_close"/>

        <ImageView
            android:id="@+id/switch_camera_iv"
            android:layout_width="40dp"
            android:layout_height="40dp"
            android:layout_weight="1"
            android:padding="8dp"
            android:src="@mipmap/ic_switch_camera"/>

        <ImageView
            android:id="@+id/compare_iv"
            android:layout_width="40dp"
            android:layout_height="40dp"
            android:layout_marginRight="10dp"
            android:padding="8dp"
            android:src="@mipmap/ic_compare"/>
    LinearLayout>

    <FrameLayout
        android:layout_width="match_parent"
        android:layout_height="0dp"
        android:layout_weight="1">

        <jp.co.cyberagent.android.gpuimage.GPUImageView
            android:id="@+id/gpuimage"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_gravity="center"/>

        <SeekBar
            android:id="@+id/tone_seekbar"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_gravity="bottom"
            android:background="#55ffffff"
            android:max="100"
            android:padding="10dp"
            android:visibility="gone"/>

    FrameLayout>

    <RelativeLayout
        android:layout_width="match_parent"
        android:layout_height="64dp"
        android:gravity="center_vertical"
        android:orientation="horizontal"
        android:padding="10dp">

        <TextView
            android:id="@+id/filter_name_tv"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_centerInParent="true"
            android:text="@string/choose_filter"
            android:textColor="@android:color/white"
            android:textSize="18sp"/>

        <ImageView
            android:id="@+id/save_iv"
            android:layout_width="36dp"
            android:layout_height="36dp"
            android:layout_alignParentEnd="true"
            android:layout_centerVertical="true"
            android:padding="5dp"
            android:src="@mipmap/ic_ok"/>
    RelativeLayout>
LinearLayout>

除了一些常规的控件外，还使用到一个叫做 GPUImageView 的自定义控件作为显示，这也是我们使用 GPUImage 最常接触的类之一。

2. CameraActivity

public class CameraActivity extends BaseActivity implements View.OnClickListener {

    private GPUImageView mGPUImageView;
    private SeekBar mSeekBar;
    private TextView mFilterNameTv;

    private GPUImageFilter mNoImageFilter = new GPUImageFilter();
    private GPUImageFilter mCurrentImageFilter = mNoImageFilter;
    private GPUImageFilterTools.FilterAdjuster mFilterAdjuster;

    private CameraLoader mCameraLoader;

    @Override
    protected void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_camera);
        initView();
        initCamera();
    }

    private void initView() {
        mGPUImageView = findViewById(R.id.gpuimage);
        mSeekBar = findViewById(R.id.tone_seekbar);
        mFilterNameTv = findViewById(R.id.filter_name_tv);
        mFilterNameTv.setOnClickListener(this);
        mSeekBar.setOnSeekBarChangeListener(mOnSeekBarChangeListener);
        findViewById(R.id.compare_iv).setOnTouchListener(mOnTouchListener);
        findViewById(R.id.close_iv).setOnClickListener(this);
        findViewById(R.id.save_iv).setOnClickListener(this);
        findViewById(R.id.switch_camera_iv).setOnClickListener(this);
    }

    private void initCamera() {
        mCameraLoader = new Camera2Loader(this);
        mCameraLoader.setOnPreviewFrameListener(new CameraLoader.OnPreviewFrameListener() {
            @Override
            public void onPreviewFrame(byte[] data, int width, int height) {
                mGPUImageView.updatePreviewFrame(data, width, height);
            }
        });
        mGPUImageView.setRatio(0.75f); // 固定使用 4:3 的尺寸
        updateGPUImageRotate();
        mGPUImageView.setRenderMode(GPUImageView.RENDERMODE_CONTINUOUSLY);
    }

    private void updateGPUImageRotate() {
        Rotation rotation = getRotation(mCameraLoader.getCameraOrientation());
        boolean flipHorizontal = false;
        boolean flipVertical = false;
        if (mCameraLoader.isFrontCamera()) { // 前置摄像头需要镜像
            if (rotation == Rotation.NORMAL || rotation == Rotation.ROTATION_180) {
                flipHorizontal = true;
            } else {
                flipVertical = true;
            }
        }
        mGPUImageView.getGPUImage().setRotation(rotation, flipHorizontal, flipVertical);
    }

    @Override
    protected void onResume() {
        super.onResume();
        if (ViewCompat.isLaidOut(mGPUImageView) && !mGPUImageView.isLayoutRequested()) {
            mCameraLoader.onResume(mGPUImageView.getWidth(), mGPUImageView.getHeight());
        } else {
            mGPUImageView.addOnLayoutChangeListener(new View.OnLayoutChangeListener() {
                @Override
                public void onLayoutChange(View v, int left, int top, int right, int bottom, int oldLeft, int oldTop,
                                           int oldRight, int oldBottom) {
                    mGPUImageView.removeOnLayoutChangeListener(this);
                    mCameraLoader.onResume(mGPUImageView.getWidth(), mGPUImageView.getHeight());
                }
            });
        }
    }

    @Override
    protected void onPause() {
        super.onPause();
        mCameraLoader.onPause();
    }

    @Override
    public void onClick(View v) {
        switch (v.getId()) {
            case R.id.filter_name_tv:
                GPUImageFilterTools.showDialog(this, mOnGpuImageFilterChosenListener);
                break;
            case R.id.close_iv:
                finish();
                break;
            case R.id.save_iv:
                saveSnapshot();
                break;
            case R.id.switch_camera_iv:
                mGPUImageView.getGPUImage().deleteImage();
                mCameraLoader.switchCamera();
                updateGPUImageRotate();
                break;
        }
    }

    private void saveSnapshot() {
        String fileName = System.currentTimeMillis() + ".jpg";
        mGPUImageView.saveToPictures("GPUImage", fileName, mOnPictureSavedListener);
    }

    private View.OnTouchListener mOnTouchListener = new View.OnTouchListener() {
        @Override
        public boolean onTouch(View v, MotionEvent event) {
            if (v.getId() == R.id.compare_iv) {
                switch (event.getAction()) {
                    case MotionEvent.ACTION_DOWN:
                        mGPUImageView.setFilter(mNoImageFilter);
                        break;
                    case MotionEvent.ACTION_UP:
                        mGPUImageView.setFilter(mCurrentImageFilter);
                        break;
                }
            }
            return true;
        }
    };

    private OnGpuImageFilterChosenListener mOnGpuImageFilterChosenListener = new OnGpuImageFilterChosenListener() {
        @Override
        public void onGpuImageFilterChosenListener(GPUImageFilter filter, String filterName) {
            switchFilterTo(filter);
            mFilterNameTv.setText(filterName);
        }
    };

    private void switchFilterTo(GPUImageFilter filter) {
        if (mCurrentImageFilter == null
                || (filter != null && !mCurrentImageFilter.getClass().equals(filter.getClass()))) {
            mCurrentImageFilter = filter;
            mGPUImageView.setFilter(mCurrentImageFilter);
            mFilterAdjuster = new GPUImageFilterTools.FilterAdjuster(mCurrentImageFilter);
            mSeekBar.setVisibility(mFilterAdjuster.canAdjust() ? View.VISIBLE : View.GONE);
        } else {
            mSeekBar.setVisibility(View.GONE);
        }
    }

    private SeekBar.OnSeekBarChangeListener mOnSeekBarChangeListener = new SeekBar.OnSeekBarChangeListener() {
        @Override
        public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) {
            if (mFilterAdjuster != null) {
                mFilterAdjuster.adjust(progress);
            }
            mGPUImageView.requestRender();
        }
        @Override
        public void onStartTrackingTouch(SeekBar seekBar) {}
        @Override
        public void onStopTrackingTouch(SeekBar seekBar) {}
    };

    private GPUImageView.OnPictureSavedListener mOnPictureSavedListener = new GPUImageView.OnPictureSavedListener() {
        @Override
        public void onPictureSaved(Uri uri) {
            String filePath = FileUtils.getRealFilePath(CameraActivity.this, uri);
            Log.d(TAG, "save to " + filePath);
            Toast.makeText(CameraActivity.this, "Saved: " + filePath, Toast.LENGTH_SHORT).show();
        }
    };

    private Rotation getRotation(int orientation) {
        switch (orientation) {
            case 90:
                return Rotation.ROTATION_90;
            case 180:
                return Rotation.ROTATION_180;
            case 270:
                return Rotation.ROTATION_270;
            default:
                return Rotation.NORMAL;
        }
    }
}

除去一些事件外，主要是通过 CameraLoader 的 OnPreviewFrameListener 回调来获取帧数据并更新。

并且，我们只需要通过切换不同的 GPUImageFilter 就可以实现不同的滤镜效果了，非常方便。

3. CameraLoader

相机操作类，抽象类，为可能需要使用的 Camera1 做准备。

public abstract class CameraLoader {

    protected OnPreviewFrameListener mOnPreviewFrameListener;

    public abstract void onResume(int width, int height);

    public abstract void onPause();

    public abstract void switchCamera();

    public abstract int getCameraOrientation();

    public abstract boolean hasMultipleCamera();

    public abstract boolean isFrontCamera();

    public void setOnPreviewFrameListener(OnPreviewFrameListener onPreviewFrameListener) {
        mOnPreviewFrameListener = onPreviewFrameListener;
    }

    public interface OnPreviewFrameListener {
        void onPreviewFrame(byte[] data, int width, int height);
    }

}

4. Camera2Loader

继承自 CameraLoader，并使用 Camera2 的相关 API 完成相机的操作。

public class Camera2Loader extends CameraLoader {

    private static final String TAG = "Camera2Loader";

    private Activity mActivity;

    private CameraManager mCameraManager;
    private CameraCharacteristics mCharacteristics;
    private CameraDevice mCameraDevice;
    private CameraCaptureSession mCaptureSession;
    private ImageReader mImageReader;

    private String mCameraId;
    private int mCameraFacing = CameraCharacteristics.LENS_FACING_BACK;
    private int mViewWidth;
    private int mViewHeight;
    private float mAspectRatio = 0.75f; // 4:3

    public Camera2Loader(Activity activity) {
        mActivity = activity;
        mCameraManager = (CameraManager) mActivity.getSystemService(Context.CAMERA_SERVICE);
    }

    @Override
    public void onResume(int width, int height) {
        mViewWidth = width;
        mViewHeight = height;
        setUpCamera();
    }

    @Override
    public void onPause() {
        releaseCamera();
    }

    @Override
    public void switchCamera() {
        mCameraFacing ^= 1;
        Log.d(TAG, "current camera facing is: " + mCameraFacing);
        releaseCamera();
        setUpCamera();
    }

    @Override
    public int getCameraOrientation() {
        int degrees = mActivity.getWindowManager().getDefaultDisplay().getRotation();
        switch (degrees) {
            case Surface.ROTATION_0:
                degrees = 0;
                break;
            case Surface.ROTATION_90:
                degrees = 90;
                break;
            case Surface.ROTATION_180:
                degrees = 180;
                break;
            case Surface.ROTATION_270:
                degrees = 270;
                break;
            default:
                degrees = 0;
                break;
        }
        int orientation = 0;
        try {
            String cameraId = getCameraId(mCameraFacing);
            CameraCharacteristics characteristics = mCameraManager.getCameraCharacteristics(cameraId);
            orientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
        Log.d(TAG, "degrees: " + degrees + ", orientation: " + orientation + ", mCameraFacing: " + mCameraFacing);
        if (mCameraFacing == CameraCharacteristics.LENS_FACING_FRONT) {
            return (orientation + degrees) % 360;
        } else {
            return (orientation - degrees) % 360;
        }
    }

    @Override
    public boolean hasMultipleCamera() {
        try {
            int size = mCameraManager.getCameraIdList().length;
            return size > 1;
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
        return false;
    }

    @Override
    public boolean isFrontCamera() {
        return mCameraFacing == CameraCharacteristics.LENS_FACING_FRONT;
    }

    @SuppressLint("MissingPermission")
    private void setUpCamera() {
        try {
            mCameraId = getCameraId(mCameraFacing);
            mCharacteristics = mCameraManager.getCameraCharacteristics(mCameraId);
            mCameraManager.openCamera(mCameraId, mCameraDeviceCallback, null);
        } catch (CameraAccessException e) {
            Log.e(TAG, "Opening camera (ID: " + mCameraId + ") failed.");
            e.printStackTrace();
        }
    }

    private void releaseCamera() {
        if (mCaptureSession != null) {
            mCaptureSession.close();
            mCaptureSession = null;
        }
        if (mCameraDevice != null) {
            mCameraDevice.close();
            mCameraDevice = null;
        }
        if (mImageReader != null) {
            mImageReader.close();
            mImageReader = null;
        }
    }

    private String getCameraId(int facing) throws CameraAccessException {
        for (String cameraId : mCameraManager.getCameraIdList()) {
            if (mCameraManager.getCameraCharacteristics(cameraId).get(CameraCharacteristics.LENS_FACING) ==
                    facing) {
                return cameraId;
            }
        }
        // default return
        return Integer.toString(facing);
    }

    private void startCaptureSession() {
        Size size = chooseOptimalSize();
        Log.d(TAG, "size: " + size.toString());
        mImageReader = ImageReader.newInstance(size.getWidth(), size.getHeight(), ImageFormat.YUV_420_888, 2);
        mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
            @Override
            public void onImageAvailable(ImageReader reader) {
                if (reader != null) {
                    Image image = reader.acquireNextImage();
                    if (image != null) {
                        if (mOnPreviewFrameListener != null) {
                            byte[] data = ImageUtils.generateNV21Data(image);
                            mOnPreviewFrameListener.onPreviewFrame(data, image.getWidth(), image.getHeight());
                        }
                        image.close();
                    }
                }
            }
        }, null);
        try {
            mCameraDevice.createCaptureSession(Arrays.asList(mImageReader.getSurface()), mCaptureStateCallback, null);
        } catch (CameraAccessException e) {
            e.printStackTrace();
            Log.e(TAG, "Failed to start camera session");
        }
    }

    private Size chooseOptimalSize() {
        Log.d(TAG, "viewWidth: " + mViewWidth + ", viewHeight: " + mViewHeight);
        if (mViewWidth == 0 || mViewHeight == 0) {
            return new Size(0, 0);
        }
        StreamConfigurationMap map = mCharacteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
        Size[] sizes = map.getOutputSizes(ImageFormat.YUV_420_888);
        int orientation = getCameraOrientation();
        boolean swapRotation = orientation == 90 || orientation == 270;
        int width = swapRotation ? mViewHeight : mViewWidth;
        int height = swapRotation ? mViewWidth : mViewHeight;
        return getSuitableSize(sizes, width, height, mAspectRatio);
    }

    private Size getSuitableSize(Size[] sizes, int width, int height, float aspectRatio) {
        int minDelta = Integer.MAX_VALUE;
        int index = 0;
        Log.d(TAG, "getSuitableSize. aspectRatio: " + aspectRatio);
        for (int i = 0; i < sizes.length; i++) {
            Size size = sizes[i];
            // 先判断比例是否相等
            if (size.getWidth() * aspectRatio == size.getHeight()) {
                int delta = Math.abs(width - size.getWidth());
                if (delta == 0) {
                    return size;
                }
                if (minDelta > delta) {
                    minDelta = delta;
                    index = i;
                }
            }
        }
        return sizes[index];
    }

    private CameraDevice.StateCallback mCameraDeviceCallback = new CameraDevice.StateCallback() {
        @Override
        public void onOpened(@NonNull CameraDevice camera) {
            mCameraDevice = camera;
            startCaptureSession();
        }
        @Override
        public void onDisconnected(@NonNull CameraDevice camera) {
            mCameraDevice.close();
            mCameraDevice = null;
        }
        @Override
        public void onError(@NonNull CameraDevice camera, int error) {
            mCameraDevice.close();
            mCameraDevice = null;
        }
    };

    private CameraCaptureSession.StateCallback mCaptureStateCallback = new CameraCaptureSession.StateCallback() {
        @Override
        public void onConfigured(@NonNull CameraCaptureSession session) {
            if (mCameraDevice == null) {
                return;
            }
            mCaptureSession = session;
            try {
                CaptureRequest.Builder builder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
                builder.addTarget(mImageReader.getSurface());
                builder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
                builder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_AUTO_FLASH);
                session.setRepeatingRequest(builder.build(), null, null);
            } catch (CameraAccessException e) {
                e.printStackTrace();
            }
        }
        @Override
        public void onConfigureFailed(@NonNull CameraCaptureSession session) {
            Log.e(TAG, "Failed to configure capture session.");
        }
    };

关于相机的使用，其实也有很多可以优化的地方，例子中从简了（例如分辨率的选择只考虑了4:3的比例，也没有使用后台线程执行一些任务，相机的一些参数调整也没有过多设置）。

5. 其余类

其余都是一些工具类了，可以在工程地址中找吧。

五、工程地址

下面是完整的工程代码，可直接在 Android Studio 上运行。

https://github.com/afei-cn/GPUImageDemo

六、GPUImage 类介绍

1. 目录结构

|— filter ： 这个包下面是各种滤镜效果类。
|— util ： 这个包下面是一些工具类。
|— GLTextureView ： 继承自 TextureView，和 GLSurfaceView 功能类似。
|— GPUImage ： 核心实现类，配合 GLSurfaceView/GLTextureView 和 GPUImageFilter 实现渲染。
|— GPUImageNativeLibrary ： 包含一些图片转码的 native 方法。
|— GPUImageRenderer ： 实际的渲染者。
|— GPUImageView ： 继承自 FrameLayout，封装了一个 GPUImage 和 GPUImageFilter，使用起来更方便。

2. 简要使用

GPUImage:

@Override
public void onCreate(final Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity);

    Uri imageUri = ...;
    gpuImage = new GPUImage(this);
    gpuImage.setGLSurfaceView((GLSurfaceView) findViewById(R.id.surfaceView));
    gpuImage.setImage(imageUri); // this loads image on the current thread, should be run in a thread
    gpuImage.setFilter(new GPUImageSepiaFilter());

    // Later when image should be saved saved:
    gpuImage.saveToPictures("GPUImage", "ImageWithFilter.jpg", null);
}

GPUImageView:

<jp.co.cyberagent.android.gpuimage.GPUImageView
    android:id="@+id/gpuimageview"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:gpuimage_show_loading="false"
    app:gpuimage_surface_type="texture_view" /> 

@Override
public void onCreate(final Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity);

    Uri imageUri = ...;
    gpuImageView = findViewById(R.id.gpuimageview);
    gpuImageView.setImage(imageUri); // this loads image on the current thread, should be run in a thread
    gpuImageView.setFilter(new GPUImageSepiaFilter());

    // Later when image should be saved saved:
    gpuImageView.saveToPictures("GPUImage", "ImageWithFilter.jpg", null);
}