Android使用Arrcface打开相机进行人脸识别

上一张效果图,渣画质,能看就好


在这里插入图片描述

功能说明:

人脸识别使用的是虹软的FreeSDK,包含人脸追踪,人脸检测,人脸识别,年龄、性别检测功能,其中本demo只使用了FT和FR(人脸追踪和人脸识别),封装了开启相机和人脸追踪、识别功能在FaceCameraHelper中。

实现逻辑:

打开相机,监听预览数据回调进行人脸追踪,且为每个检测到的人脸都分配一个trackID(上下帧位置变化不大的人脸框可认为是同一个人脸,具体实现的逻辑可见代码),同时,为了人脸搜索,为每个trackID都分配一个状态(识别中,识别失败,识别通过)、姓名,识别通过则在人脸框上显示姓名,否则只显示trackID(本demo没配服务端,只做了模拟操作)。流程说明见下图。
Android使用Arrcface打开相机进行人脸识别_第1张图片
在这里插入图片描述

FaceCameraHelper包含的接口:


 public interface FaceTrackListener {
 
        /**
         * 回传相机预览数据和人脸框位置
         *
         * @param nv21        相机预览数据
         * @param ftFaceList  待处理的人脸列表
         * @param trackIdList 人脸追踪ID列表
         */
        void onPreviewData(byte[] nv21, List ftFaceList, List trackIdList);
 
 
        /**
         * 当出现异常时执行
         *
         * @param e 异常信息
         */
        void onFail(Exception e);
 
 
        /**
         * 当相机打开时执行
         *
         * @param camera 相机实例
         */
        void onCameraOpened(Camera camera);
 
        /**
         * 根据自己的需要可以删除部分人脸,比如指定区域、留下最大人脸等
         *
         * @param ftFaceList  人脸列表
         * @param trackIdList 人脸追踪ID列表
         */
        void adjustFaceRectList(List ftFaceList, List trackIdList);
 
        /**
         * 请求人脸特征后的回调
         *
         * @param frFace    人脸特征数据
         * @param requestId 请求码
         */
        void onFaceFeatureInfoGet(@Nullable AFR_FSDKFace frFace, Integer requestId);
    }

FT人脸框绘制并回调数据:


@Override
    public void onPreviewFrame(byte[] nv21, Camera camera) {
        if (faceTrackListener != null) {
            ftFaceList.clear();
            int ftCode = ftEngine.AFT_FSDK_FaceFeatureDetect(nv21, previewSize.width, previewSize.height, AFT_FSDKEngine.CP_PAF_NV21, ftFaceList).getCode();
            if (ftCode != 0) {
                faceTrackListener.onFail(new Exception("ft failed,code is " + ftCode));
            }
            refreshTrackId(ftFaceList);
            faceTrackListener.adjustFaceRectList(ftFaceList, currentTrackIdList);
            if (surfaceViewRect != null) {
                Canvas canvas = surfaceViewRect.getHolder().lockCanvas();
                if (canvas == null) {
                    faceTrackListener.onFail(new Exception("can not get canvas of surfaceViewRect"));
                    return;
                }
                canvas.drawColor(0, PorterDuff.Mode.CLEAR);
                if (ftFaceList.size() > 0) {
                    for (int i = 0; i < ftFaceList.size(); i++) {
                        Rect adjustedRect = TrackUtil.adjustRect(new Rect(ftFaceList.get(i).getRect()), previewSize.width, previewSize.height, surfaceWidth, surfaceHeight, cameraOrientation, mCameraId);
                        TrackUtil.drawFaceRect(canvas, adjustedRect, faceRectColor, faceRectThickness, currentTrackIdList.get(i), nameMap.get(currentTrackIdList.get(i)));
                    }
                }
                surfaceViewRect.getHolder().unlockCanvasAndPost(canvas);
            }
 
            faceTrackListener.onPreviewData(nv21, ftFaceList, currentTrackIdList);
        }
    }

大多数设备相机预览数据图像的朝向在横屏时为0度。其他情况按逆时针依次增加90度,因此人脸框的绘制需要做同步转化。CameraID为0时,也就是后置摄像头情况,相机预览数据的显示为原画面,而CameraID为1时,也就是前置摄像头情况,相机的预览画面显示为镜像画面,适配的代码:


/**
     * @param rect          FT人脸框
     * @param previewWidth  相机预览的宽度
     * @param previewHeight 相机预览高度
     * @param canvasWidth   画布的宽度
     * @param canvasHeight  画布的高度
     * @param cameraOri     相机预览方向
     * @param mCameraId     相机ID
     * @return
     */
    static Rect adjustRect(Rect rect, int previewWidth, int previewHeight, int canvasWidth, int canvasHeight, int cameraOri, int mCameraId) {
        if (rect == null) {
            return null;
        }
        if (canvasWidth < canvasHeight) {
            int t = previewHeight;
            previewHeight = previewWidth;
            previewWidth = t;
        }
 
        float horizontalRatio;
        float verticalRatio;
        if (cameraOri == 0 || cameraOri == 180) {
            horizontalRatio = (float) canvasWidth / (float) previewWidth;
            verticalRatio = (float) canvasHeight / (float) previewHeight;
        } else {
            horizontalRatio = (float) canvasHeight / (float) previewHeight;
            verticalRatio = (float) canvasWidth / (float) previewWidth;
        }
        rect.left *= horizontalRatio;
        rect.right *= horizontalRatio;
        rect.top *= verticalRatio;
        rect.bottom *= verticalRatio;
 
        Rect newRect = new Rect();
 
        switch (cameraOri) {
            case 0:
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.left = canvasWidth - rect.right;
                    newRect.right = canvasWidth - rect.left;
                } else {
                    newRect.left = rect.left;
                    newRect.right = rect.right;
                }
                newRect.top = rect.top;
                newRect.bottom = rect.bottom;
                break;
            case 90:
                newRect.right = canvasWidth - rect.top;
                newRect.left = canvasWidth - rect.bottom;
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.top = canvasHeight - rect.right;
                    newRect.bottom = canvasHeight - rect.left;
                } else {
                    newRect.top = rect.left;
                    newRect.bottom = rect.right;
                }
                break;
            case 180:
                newRect.top = canvasHeight - rect.bottom;
                newRect.bottom = canvasHeight - rect.top;
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.left = rect.left;
                    newRect.right = rect.right;
                } else {
                    newRect.left = canvasWidth - rect.right;
                    newRect.right = canvasWidth - rect.left;
                }
                break;
            case 270:
                newRect.left = rect.top;
                newRect.right = rect.bottom;
                if (mCameraId == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    newRect.top = rect.left;
                    newRect.bottom = rect.right;
                } else {
                    newRect.top = canvasHeight - rect.right;
                    newRect.bottom = canvasHeight - rect.left;
                }
                break;
            default:
                break;
        }
        return newRect;
    }

由于FR引擎不支持多线程调用,因此只能串行执行,若需要更高效的实现,可创建多个FREngine实例进行任务分配。

FR线程队列:


     private LinkedBlockingQueue faceRecognizeRunnables = new LinkedBlockingQueue(MAX_FRTHREAD_COUNT);

FR线程:

public class FaceRecognizeRunnable implements Runnable {
        private Rect faceRect;
        private int width;
        private int height;
        private int format;
        private int ori;
        private Integer requestId;
        private byte[]nv21Data;
        public FaceRecognizeRunnable(byte[]nv21Data,Rect faceRect, int width, int height, int format, int ori, Integer requestId) {
            if (nv21Data==null) {
                return;
            }
            this.nv21Data = new byte[nv21Data.length];
            System.arraycopy(nv21Data,0,this.nv21Data,0,nv21Data.length);
            this.faceRect = new Rect(faceRect);
            this.width = width;
            this.height = height;
            this.format = format;
            this.ori = ori;
            this.requestId = requestId;
        }
 
        @Override
        public void run() {
            if (faceTrackListener!=null && nv21Data!=null) {
                if (frEngine != null) {
                    AFR_FSDKFace frFace = new AFR_FSDKFace();
                    int frCode = frEngine.AFR_FSDK_ExtractFRFeature(nv21Data, width, height, format, faceRect, ori, frFace).getCode();
                    if (frCode == 0) {
                        faceTrackListener.onFaceFeatureInfoGet(frFace, requestId);
                    } else {
                        faceTrackListener.onFaceFeatureInfoGet(null, requestId);
                        faceTrackListener.onFail(new Exception("fr failed errorCode is " + frCode));
                    }
                    nv21Data = null;
                }else {
                    faceTrackListener.onFaceFeatureInfoGet(null, requestId);
                    faceTrackListener.onFail(new Exception("fr failed ,frEngine is null" ));
                }
                if (faceRecognizeRunnables.size()>0){
                    executor.execute(faceRecognizeRunnables.poll());
                }
            }
        }
    }

上下帧是否为相同人脸的判断(trackID刷新):


/**
     * 刷新trackId
     *
     * @param ftFaceList 传入的人脸列表
     */
    public void refreshTrackId(List ftFaceList) {
        currentTrackIdList.clear();
        //每项预先填充-1
        for (int i = 0; i < ftFaceList.size(); i++) {
            currentTrackIdList.add(-1);
        }
        //前一次无人脸现在有人脸,填充新增TrackId
        if (formerTrackIdList.size() == 0) {
            for (int i = 0; i < ftFaceList.size(); i++) {
                currentTrackIdList.set(i, ++currentTrackId);
            }
        } else {
            //前后都有人脸,对于每一个人脸框
            for (int i = 0; i < ftFaceList.size(); i++) {
                //遍历上一次人脸框
                for (int j = 0; j < formerFaceRectList.size(); j++) {
                    //若是同一张人脸
                    if (TrackUtil.isSameFace(SIMILARITY_RECT, formerFaceRectList.get(j), ftFaceList.get(i).getRect())) {
                        //记录ID
                        currentTrackIdList.set(i, formerTrackIdList.get(j));
                        break;
                    }
                }
            }
        }
        //上一次人脸框不存在此人脸
        for (int i = 0; i < currentTrackIdList.size(); i++) {
            if (currentTrackIdList.get(i) == -1) {
                currentTrackIdList.set(i, ++currentTrackId);
            }
        }
        formerTrackIdList.clear();
        formerFaceRectList.clear();
        for (int i = 0; i < ftFaceList.size(); i++) {
            formerFaceRectList.add(new Rect(ftFaceList.get(i).getRect()));
            formerTrackIdList.add(currentTrackIdList.get(i));
        }
    } 

项目地址:https://github.com/wangshengyang1996/FaceTrackDemo

若有不当的地方望指出。

你可能感兴趣的:(Android使用Arrcface打开相机进行人脸识别)