人脸识别SeetaFace2原理与代码详解
人脸识别SeetaFace2原理与代码详解
- 前言
- 一、人脸识别步骤
- 二、SeetaFace2基本介绍
- 三、seetaFace2人脸注册、识别代码详解
- 3.1 人脸注册
- 3.1.1 人脸检测
- 3.1.2 人脸对齐
- 3.1.3 人脸特征提取
- 3.2 人脸识别
- 附：seetaface2代码的三个接口
前言
生物特征识别（Biometrics）是利用机器学习算法对生物个体，尤其是人类本身的某些特性进行分析和识别的技术。其不仅限于利用人体所固有的指纹、掌纹、面部、虹膜等生理特征，也包括使用步态、打字习惯等个人行为进行身份鉴定。近年来广义的定义也将表情等图片属性归为软生物特征识别（Soft biometrics）一类。生物特征识别应用的领域主要是门禁系统、考勤系统等。
一、人脸识别步骤
如图所示为本人使用XMind ZEN软件绘制的人脸注册、识别流程图，从图中可以清晰的看出人脸识别的步骤。其中，最主要的模块分为3步：人脸检测，人脸对齐，人脸特征提取。
1）人脸检测（face detect）：在图像中首先定位出人脸的位置，然后裁剪(crop)出包含人脸位置的矩形框，一般还会进行填充、缩放到指定尺寸，还可能会对人脸图像进行标准化normalize；
2）人脸对齐（face alignment）：提取人脸关键点坐标，然后使用放射变化或相似变换等进行人脸对齐变换。（人脸对齐的目标就是把所有的人脸图片统一到一个固定的正脸姿态和大小，从而提高模型对人脸姿态变化的鲁棒性。）
3）人脸特征提取（extract feature）:主要使用深度学习等方法提取人脸的特征。
二、SeetaFace2基本介绍
SeetaFace2.0是中科视拓开源的第二代跨平台C++商业人脸识别库，关于其介绍可以参考以下链接，本文将不再赘述。
1、官方Github源码：
2、其他博客介绍
三、seetaFace2人脸注册、识别代码详解
主要参考serach工程中的example.cpp
3.1 人脸注册
3.1.1 人脸检测
通过example.cpp line42中 engine.Register( image )跳转到FaceDetectorPrivate.cpp中Detect函数，该函数输入一张包含人脸的图片，输出SeetaFaceInfoArray格式的包含人脸框（x,y,w,h）和人脸置信度得分score的数据，以及一幅图像中人脸的个数，并将SeetaFaceInfoArray格式的以上数据存入vector中。
SeetaFaceInfoArray FaceDetectorPrivate::Detect( const SeetaImageData &image )
{
SeetaFaceInfoArray ret;
ret.size = 0;
ret.data = nullptr;
Impl *p = ( Impl * )impl_; //??
if( !p->IsLegalImage( image ) )
{
return ret;
}
// sclae image
seeta::Image img = image;
float scale = 1;
seeta::Image scaled_img = ScaleImage( img, p->width_limit_, p->height_limit_, &scale );
img = scaled_img;
img = seeta::color( img );
int pad_h = std::min( int( p->max_pad_ratio * img.height() ), p->max_pad_h );
int pad_w = std::min( int( p->max_pad_ratio * img.width() ), p->max_pad_w );
SeetaImageData img_pad;
img_pad.width = img.width() + 2 * pad_w;
img_pad.height = img.height() + 2 * pad_h;
img_pad.channels = img.channels();
img_pad.data = new uint8_t[img_pad.channels * img_pad.height * img_pad.width];
p->PadImage( img.data(), img.width(), img.height(), img.channels(), img_pad.data, pad_w, pad_h );
auto local_min_face_size = std::max( 12, int( p->min_face_ * scale ) );
auto local_max_face_size = p->max_face_;
if( local_max_face_size > 0 ) local_max_face_size = std::max( 12, int( p->max_face_ * scale ) );
std::vector<Rect> winList;
winList = p->SlidingWindow( img, img_pad, p->net_[0], p->class_threshold_[0], local_min_face_size, local_max_face_size );
winList = p->NMS( winList, true, p->nms_threshold_[0] );
// std::cout << "Stage1 result: " << winList.size() << std::endl;
winList = p->RunNet( img_pad, p->net_[1], p->class_threshold_[1], 24, winList );
winList = p->NMS( winList, true, p->nms_threshold_[1] );
// std::cout << "Stage2 result: " << winList.size() << std::endl;
winList = p->RunNet( img_pad, p->net_[2], p->class_threshold_[2], 48, winList );
winList = p->NMS( winList, false, p->nms_threshold_[2] );
// std::cout << "Stage3 result: " << winList.size() << std::endl;
// scale result
for( auto &info : winList )
{
info.x -= pad_w;
info.y -= pad_h;
info.x = int( info.x / scale );
info.y = int( info.y / scale );
info.width = int( info.width / scale );
info.height = int( info.height / scale );
}
std::vector<Rect> &preList = p->preList_;
if( p->stable_ )
{
for( size_t i = 0; i < winList.size(); i++ )
{
for( size_t j = 0; j < preList.size(); j++ )
{
if( p->IoU( winList[i], preList[j] ) > 0.85 )
winList[i] = preList[j];
else
if( p->IoU( winList[i], preList[j] ) > 0.6 )
{
winList[i].x = ( winList[i].x + preList[j].x ) / 2;
winList[i].y = ( winList[i].y + preList[j].y ) / 2;
winList[i].width = ( winList[i].width + preList[j].width ) / 2;
winList[i].height = ( winList[i].height + preList[j].height ) / 2;
}
}
}
preList = winList;
}
delete[] img_pad.data;
m_pre_faces.clear();
m_pre_faces = p->TransWindow( image, image, winList );
ret.size = int(m_pre_faces.size());
ret.data = m_pre_faces.data();
return ret;
}
3.1.2 人脸对齐
返回到Register通过FaceEngin.h中line66的return this->Register( image, faces[0] );跳转到FaceLandmarkerPrivate.cppline135 的PointDetectLandmarks函数中，即为提取人脸特征标志点的函数实现
bool FaceLandmarkerPrivate::PointDetectLandmarks( const SeetaImageData &src_img, const SeetaRect &face_info, SeetaPointF *landmarks, int *masks ) const
{
// bounding box
double width = face_info.width - 1, height = face_info.height - 1;
double min_x = face_info.x, max_x = face_info.x + width;
double min_y = face_info.y, max_y = face_info.y + height;
// move bounding box
min_x += width * x_move_;
max_x += width * x_move_;
min_y += height * y_move_;
max_y += height * y_move_;
//make the bounding box square
double center_x = ( min_x + max_x ) / 2.0, center_y = ( min_y + max_y ) / 2.0;
double r = ( ( width > height ) ? width : height ) / 2.0;
min_x = center_x - r;
max_x = center_x + r;
min_y = center_y - r;
max_y = center_y + r;
width = max_x - min_x;
height = max_y - min_y;
// expand
min_x = round( min_x - width * expand_size_ );
min_y = round( min_y - height * expand_size_ );
max_x = round( max_x + width * expand_size_ );
max_y = round( max_y + height * expand_size_ );
SeetaImageData dst_img;
dst_img.width = int( max_x ) - int( min_x ) + 1;
dst_img.height = int( max_y ) - int( min_y ) + 1;
dst_img.channels = src_img.channels;
std::unique_ptr<uint8_t[]> dst_img_data( new uint8_t[( int( max_x ) - int( min_x ) + 1 ) * ( int( max_y ) - int( min_y ) + 1 ) * src_img.channels] );
dst_img.data = dst_img_data.get();
CropFace( src_img.data, src_img.width, src_img.height, src_img.channels,
dst_img.data, int( min_x ), int( min_y ), int( max_x ), int( max_y ) );
bool flag = PredictLandmark( dst_img, landmarks, masks );
for( int i = 0; i < landmark_num_; i++ )
{
landmarks[i].x += min_x;
landmarks[i].y += min_y;
}
return flag;
}
3.1.3 人脸特征提取
主要的实现在FaceRecognizerPrivate.cppline636的ExtractFeature函数中.
bool FaceRecognizerPrivate::ExtractFeature( const std::vector<SeetaImageData> &faces, float *feats, bool normalization )
{
if( !recognizer->net ) return false;
if( faces.empty() ) return true;
int number = int( faces.size() );
int channels = GetCropChannels();
int height = GetCropHeight();
int width = GetCropWidth();
auto single_image_size = channels * height * width;
std::unique_ptr<unsigned char[]> data_point_char( new unsigned char[number * single_image_size] );
for( int i = 0; i < number; ++i )
{
if( faces[i].channels == channels &&
faces[i].height == height &&
faces[i].width == width )
{
CopyData( &data_point_char[i * single_image_size], faces[i].data, single_image_size );
continue;
}
if( recognizer->method == "resize" )
{
seeta::Image face( faces[i].data, faces[i].width, faces[i].height, faces[i].channels );
seeta::Image fixed = seeta::resize( face, seeta::Size( GetCropWidth(), GetCropHeight() ) );
CopyData( &data_point_char[i * single_image_size], fixed.data(), single_image_size );
}
else
{
seeta::Image face( faces[i].data, faces[i].width, faces[i].height, faces[i].channels );
seeta::Rect rect( ( GetCropWidth() - faces[i].width ) / 2, ( GetCropHeight() - faces[i].height ) / 2, GetCropWidth(), GetCropHeight() );
seeta::Image fixed = seeta::crop_resize( face, rect, seeta::Size( GetCropWidth(), GetCropHeight() ) );
CopyData( &data_point_char[i * single_image_size], fixed.data(), single_image_size );
}
}
return LocalExtractFeature(
number, width, height, channels, data_point_char.get(),
recognizer->net, GetMaxBatch(), recognizer->header.blob_name.c_str(), GetFeatureSize(),
feats,
normalization,
recognizer->sqrt_times );
}
3.2 人脸识别
人脸检测、人脸对齐和人脸特征提取与人脸注册内容相同，不再赘述；人脸图像质量评估较为简单，在evaluate函数中；
float QualityAssessor::evaluate(const SeetaImageData &image, const SeetaRect &face,
const SeetaPointF *points) const {
// std::cout << "=============================" << std::endl;
float clarity;
if (check_lightness(image, face)
&& check_face_size(face)
&& check_pose(image, face, points)
&& check_clarity(image, face, clarity)) {
return clarity;
} else {
return 0;
}
}
人脸特征比对模块在FaceRecognizerPrivate.cppline461的CalcSimilarity函数中。
float FaceRecognizerPrivate::CalcSimilarity( const float *fc1, const float *fc2, long dim )
{
if( dim <= 0 ) dim = GetFeatureSize();
double dot = 0;
double norm1 = 0;
double norm2 = 0;
for( size_t i = 0; i < dim; ++i )
{
dot += fc1[i] * fc2[i];
norm1 += fc1[i] * fc1[i];
norm2 += fc2[i] * fc2[i];
}
double similar = dot / ( sqrt( norm1 * norm2 ) + 1e-5 );
return recognizer->trans( float( similar ) );
}
附：seetaface2代码的三个接口
FaceDetector：人脸检测器会对输入的彩色（三通道）或灰度（单通道）图像进行人脸检测，并返回所
有检测到的人脸位置信息。
FaceLandmarks：人脸特征点检测器根据输入的彩色图片和人脸位置，对人脸特征点进行检测，并返回对
应人脸的特定数量的特征点坐标信息。
FaceRecognizer： 人脸识别器根据输入的彩色图片和人脸特征点，对输入的人脸提取特征值，根据提取的
特征值进行两张人脸相似度的比较。