Ubunutu18.04+Qt5.14+Dlib19.24+Opencv3.4.16实时人眼监测实验

---

1 前言

  在Ubuntu18.04实现的一个人眼监测小程序，使用Qt5.14、Dlib19.24、Opencv3.4.16实现。
  其主要实现思想是，首先通过Opencv获取摄像头数据,然后通过Dlib提取人脸68关键点的算法对所输入图片中的人脸进行关键点检测，检测出眼部的关键点后,在用Opnecv中的画图命令进行绘制，再利用Qt进行界面展示。

---

2 效果

  人眼监测小程序效果如下图所示，将画面中的人眼处用绿色的线进行框选。

---

3 Ubuntu18.04下Qt、Opencv、Dlib的配置

3.0 Ubuntu18.04的安装以及一些基本配置

  U此部分涉及到buntu18.04系统的安装、配置一些基础的应用软件等过程。
  (1)引导盘的制作;(2)安装ubuntu18.04系统;(3)安装搜狗输入法;(4)安装百度网盘;(5)安装Chrome浏览器;(6)安装激活Pycharm;(7)安装VScode;
  整体的大概流程参照这篇博客

3.1 Qt

  从Qt官网下载链接https://download.qt.io/archive/qt/处下载所对应版本,我这里直接下载的是https://download.qt.io/archive/qt/5.14/5.14.2/这个页面下的这个qt-opensource-linux-x64-5.14.2.run文件.下载下来后直接命令行运行,按照步骤走即可,注意一下安装Qt的具体位置.

3.2 Opencv

  从github上下载Opencv的源码进行编译,我这次没下新版本,因为之前有用过这个Opnecv3.4.16,所以就下了这个https://github.com/opencv/opencv/tree/3.4.16版本,没有什么特别的用意,如果只是做一些小demo,仅仅用到一些opencv的基础操作如读取图片等操作,下哪个版本的都可以.
  在用到cmake-gui配置编译选项的时候,勾选编译opencv_world,这样的话所有的库文件全部打包成一个,可以简便pro文件的写法,也不用去查某个函数到底对应哪个库,这个libopencv_world.so.3.4.16也就20多MB吧.(此处是相对而言的,也可以不编译为一个库)

3.3 Dlib

  从github上下载Dlib的源码https://github.com/davisking/dlib进行编译,过程与编译opencv时类似.

4 核心代码

  本项目下的文件层级如下图所示.

4.1 pro文件

QT       += core gui

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

CONFIG += c++14

# You can make your code fail to compile if it uses deprecated APIs.
# In order to do so, uncomment the following line.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000    # disables all the APIs deprecated before Qt 6.0.0

SOURCES += \
    camera.cpp \
    fatiguedetect.cpp \
    main.cpp \
    widget.cpp

HEADERS += \
    camera.h \
    fatiguedetect.h \
    widget.h

FORMS += \
    widget.ui

#opencv库的路径opencv3.4.16
INCLUDEPATH += /home/ai/software/opencv3416_installed/include
LIBS += /home/ai/software/opencv3416_installed/lib/libopencv_world.so

# dlib19.24库的路径
INCLUDEPATH += /home/ai/software/dlib_installed/include
LIBS += /home/ai/software/dlib_installed/lib/libdlib.a

# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

DISTFILES +=

4.2 Widget.cpp

#include "widget.h"
#include "ui_widget.h"
#include 


Widget::Widget(QWidget *parent)
    : QWidget(parent)
    , ui(new Ui::Widget)
{
    ui->setupUi(this);
    camera_thread = new Camera();
    camera_thread->set_cam_number(0);
    connect(camera_thread, SIGNAL(sendPicture(QImage)),this,SLOT(receive(QImage)));
    camera_thread->open_camera();
}

Widget::~Widget()
{
    delete ui;
}


void Widget::receive(QImage img)
{
    QImage scaled_img = img.scaled(ui->label->width(), ui->label->height());
    ui->label->setPixmap(QPixmap::fromImage(scaled_img));
}

4.3 fatiguedetect.cpp

#include "fatiguedetect.h"

// 常量定义
const double close_standard = 0.45;
const int eye_l_len = 60;

// 判断状态的类
class JudgeCondition : public QObject {
    Q_OBJECT

public:
    explicit JudgeCondition(QObject *parent = nullptr) : QObject(parent), condition(0), heavy_flag(0) {};

    void judge(std::deque<int> &left_eye_l, std::deque<int> &right_eye_l)
    {
        int left_sum = std::accumulate(left_eye_l.begin(), left_eye_l.end(), 0);
        int right_sum = std::accumulate(right_eye_l.begin(), right_eye_l.end(), 0);

        if (left_sum == eye_l_len && right_sum == eye_l_len) {
            ++heavy_flag;
            if (heavy_flag > 100) {
                condition = 3;
            } else {
                condition = 2;
            }
        } else {
            heavy_flag = 0;
            condition = 1;
        }

        emit sendCondition(condition);
    };

signals:
    void sendCondition(int);

private:
    int condition;  // 状态：0=图像缺失 1=清醒 2=疲劳 3=重度疲劳
    int heavy_flag;  // 重度疲劳标志位
};


FatigueDetect::FatigueDetect(QObject *parent) : QObject(parent),
    noImageFlag(0),
    noImage(0)
{
    cout << "[INFO] loading facial landmark predictor..." << endl;
    dlib::deserialize("./model.dat") >> predictor;          //是直接调试运行,则在build文件夹内;如果为release则与.exe在同一文件夹内
    detector = dlib::get_frontal_face_detector();
}

cv::Mat FatigueDetect::detect(cv::Mat frame)
{
    std::vector<std::vector<cv::Point>> pos;
    cv::Mat resized_frame;
    cv::resize(frame, resized_frame, cv::Size(720, 720 * frame.rows / frame.cols));
    cv::Mat gray;
//    cv::cvtColor(resized_frame, gray, cv::COLOR_BGR2GRAY);

    cv_image<bgr_pixel> dlib_img(resized_frame);

    std::vector<dlib::rectangle> rects = detector(dlib_img, 0);    //这里放入detector的必须是dlib类型参数，不能是cv::Mat，不报错，但编译时出“问题”

    if (rects.size() == 0)
    {
        noImage = 1;
        emit noSignal(noImageFlag);
    }

    for (dlib::rectangle rect : rects)    //for(类型 变量名称 : 容器/数组等内容)
    {
        noImage = 0;

        dlib::full_object_detection shape = predictor(dlib_img, rect);

        std::vector<cv::Point> pos_;

        for (int i = 36; i <= 41; i++)   //左眼关键点标识
        {
            cv::Point point(shape.part(i).x(), shape.part(i).y());
            pos_.push_back(point);
        }

        for (int i = 42; i <= 47; i++)   //右眼关键点标识
        {
            cv::Point point(shape.part(i).x(), shape.part(i).y());
            pos_.push_back(point);
        }

        pos.push_back(pos_);

        draw_point_and_line(pos_, resized_frame);
    }

    cv::Mat resized_back_frame;
    cv::resize(resized_frame, resized_back_frame, cv::Size(frame.cols, frame.rows));
    return resized_back_frame;
}

void FatigueDetect::draw_point_and_line(std::vector<cv::Point> pos_, cv::Mat &image)
{
    int linewidth = 1;
    for (int i = 0; i < 5; i++)
    {
       cv::line(image, pos_[i], pos_[i + 1], cv::Scalar(0, 255, 0), linewidth);
    }
    cv::line(image, pos_[0], pos_[5], cv::Scalar(0, 255, 0), linewidth);
    for (int i = 6; i < 11; i++)
    {
       cv::line(image, pos_[i], pos_[i + 1], cv::Scalar(0, 255, 0), linewidth);
    }
    cv::line(image, pos_[6], pos_[11], cv::Scalar(0, 255, 0), linewidth);
    for (cv::Point point : pos_)
    {
      cv::circle(image, point, linewidth, cv::Scalar(255, 0, 0), -1);
    }
}

5 资源下载

本案例中涉及到的工程文件到此处下载。