人脸识别Python教学

1.摘要

人脸识别在 Computer Vision 中一直是很火热的话题，也是目前广为人知的一项技术。本质上分为 Face Verification、Face Recognition：前者为验证两张人脸是否为同一个人，属于一对一的过程；后者则是从数据库里辨识出相同的人脸，属于一对多的过程。详细的人脸辨识解说可以参考: 使用深度学习进行人脸辨识: Triplet loss, Large margin loss(ArcFace)。

2.实现过程

本文将要使用 Python 来进行人脸识别的实现，过程分为几个阶段:

• Face Detection
• Face Align
• Feature Extraction
• Create Database
• Face Recognition

2.0安装相关库

首先安装相关库

pip install scikit-learn
pip install onnxruntime

2.1 Face Detection

这部分要进行人脸检测，可以使用Python API MTCNN、Retina Face这边示范使用RetinaFace来进行检测。

• 安装RetinaFace

pip install retinaface

• 检测
  接着就可以来检测人脸啦，输出包含人脸预测框的左上角跟右下角点、两个眼睛、鼻子、嘴巴两边的坐标值：

import cv2
from retinaface import RetinaFace

detector = RetinaFace(quality="normal")
img_path = "001.jpg"
img_bgr = cv2.imread(img_path, cv2.IMREAD_COLOR)
img_rgb = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)
detections = detector.predict(img_bgr)
print(detections)
# # output[{‘x1’: 243, ‘y1’: 142, ‘x2’: 557, ‘y2’: 586, ‘left_eye’: (303, 305), ‘right_eye’: (431, 346), ‘nose’: (305, 403), ‘left_lip’: (272, 468), ‘right_lip’: (364, 505)}]
img_result = detector.draw(img_rgb, detections)
img = cv2.cvt_color(img_result, cv2.COLOR_RGB2BGR)
cv2.imshow("windows", img)
key = cv2,waitKey() & 0xffff
if key==ord("q"):
	print("exit")
cv2.destoryWindow("windows")
	

若使用RetinaFace的时候，出现以下错误：

有可能是因为无法导入shapely.geometry模块的关系，因此要先下去下载shapely包，下载地址https://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely 下载完成以后再执行以下指令：

pip install <your Shapely package path>

测试安装是否成功

$ python
>>> from shapely.geometry import Polygon

2.2 Face Align

这部分要将人脸特征点进行对齐，需要先定义要对齐的坐标，在onnxarcface_inference.ipynb里的Preprocess images中可以看到。
接着就用skimage套件transform.SimilarityTransform()得到要变换的矩阵，然后进行对齐：

import cv2
from retinaface import RetinaFace
from skimage import transform as trans

src = np.array([
   [30.2946, 51.6963],
   [65.5318, 51.5014],
   [48.0252, 71.7366],
   [33.5493, 92.3655],
   [62.7299, 92.2041]], dtype=np.float32)

detector = RetinaFace(quality="normal")
img_path = "001.jpg"
img_bgr = cv2.imread(img_path, cv2.IMREAD_COLOR)
img_rgb = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)
detections = detector.predict(img_bgr)

for i, face_info in enumerate(detections):
	face_position = [face_info["x1"], face_info["y1"], face_info["x2"], face_info["y2"]]
	face_landmarks = [face_info["left_eye"], face_info["right_eye"], face_info["nose"], face_info["left_lip"], face_info["right_lip"]]
	dst = np.array(face_landmarks, dtype=np.float32).shape(5, 2)
	tform = trans.SimilarityTransform()
	tform.estimate(dst, src)
	M = tform.params[0:2, :]
	aligned = cv2.warpAffine(imgRGB, M, (112, 112), borderValue=0)

对齐特征点后的人脸会呈现以下结果：

2.3 Feature extraction

这部分要提取刚刚对齐后的人脸特征，这边使用onnx ArcFace model。

• InsightFace-REST模型arcface_r100_v1下载
• onnx 官方模型下載

如果是下载onnx官方模型需要先进行更新，因为该模型的BatchNorm节点中spatial为0，参考: https://github.com/onnx/models/issues/156。不过转换过后的模型准确度比较差，因此数据集需要放两张比较能够检测出来的图片。

import onnx
model = onnx.load("model/arcfaceresnet100-8.onnx")
for node in model.graph.node:
	if(node.op_type == "BatchNormalization"):
		for attr in node.attribute:
			if(attr.name == "spatial"):
				attr.i = 1
onnx.save(model, "model/arcfaceresnet100.onnx")

接着使用模型进行特征提取：将对齐后的人脸做转置，再转换的type为float32，最后进行推理：

import numpy as np
import onnxruntime as rt
from sklearn.preprocessing import normalize

onnx_path = "model/arcfaceresnet100.onnx"
extractor = rt.InferenceSession(onnx_path)

t_aligned = np.transpose(aligned, (2, 0, 1))
inputs = t_aligned.astype(np.float32)
input_blob = np.expand_dims(inputs, axis=0)

first_input_name = extractor.get_inputs()[0].name
first_output_name = extractor.get_outputs()[0].name

predict = extractor.run([first_output_name], {first_input_name:input_blob})[0]
final_embedding = normalize(predict).flatten()

2.4 Create Database

这部分要将识别的人脸资料写进数据库里，这边数据库是使用sqlite。
首先，准备要识别的人脸资料：

接着把上面的Face Detection、Face Align、Feature extraction写成函数，调用比较方便。然后将资料夹的图片分别进行检测、对齐、提取特征后，再写入数据库中。

import sqlite3
import io
import os

def adapt_array(arr):
	out = io.BytesIO()
	np.save(out, arr)
	out.seek(0)
	return sqlite3.Binary(out.read())

def convert_array(text):
	out = io.ByteIO(text)
	out.seek(0)
	return np.load(out)

def load_file(file_path):
	file_data = {}
	for person_name in os.listdir(file_path):
		person_file = os.path.join(file_path, person_name)
		
		total_picture = []
		for picture in os.listdir(person_file):
			picture_path = os.path.join(person_file, picture)
			total_pictures.append(picture_path)
		file_data[person_name] = total_pictures
	return file_data

sqlite3.register_adapter(np.ndarray, adapt_array)
sqlite3.register_converter("ARRAY", convert_array)
conn_db = sqlite3.connect("database.db")
conn_db.execute("CREATE TABLE face_info (id INT PRIMARY KEY NOT NULL,name TEXT NOT NULL,embedding ARRAY NOT NULL)")

file_path = "database"
if os.path.exists(file_path):
	file_data = load_file(file_path)

	for i, person_name in enumerate(file_data.keys()):
		picture_path = file_data[person_name]
		sum_embeddings = np.zeros([1, 512])
		for j, picture in enumerate(picture_path):
			img_rgb, detections = face_detect(picture)
			position, landmarks, embeddings = get_embeddings(img_rgb, detections)
			sum_embeddings += embeddings
		final_embedding = sum_embeddings / len(picture_path)
		adapt_embedding = adapt_array(final_embedding)
		
		conn_db.execute("INSERT INTO face_info (id, name, embedding) VALUES (?, ?, ?)", (i, person_name, adapt_embedding))
	conn_db.commit()
	conn_db.close()

确认是否写入数据库里面：

import sqlite3
import numpy as np
import io

def adapt_array(arr):
	out = io.ByteIO()
	np.save(out, arr)
	out.seek(0)
	return sqlite3.Binary(out.read())

def convert_array(text):
	out = io.ByteIO(text)
	out.seek(0)
	return np.load(out)

sqlite3.register_adapter(np.ndarray, adapt_array)
sqlite3.register_converter("array", convert_array)
conn_db = sqlite3.connect("database.db")

cursor = conn_db.execute("SELECT * FROM face_info")
db_data = cursor.fetchall()
for data in db_data:
	print(data)
conn_db.close()

2.5 Face Recognition

这部分是要将数据库里的人脸特征跟输入照片进行比对，这里使用L2-Norm来计算之间的距离。最后再设定thresold，若L2-Norm距离大于threshold表示输入照片不为数据库中的任何一人，反之，L2-Norm距离最小的人脸与输入照片为同一人。

import numpy as np
import sqlite3
import io
import os

conn_db = sqlite3.connect(db_path)
cursor = conn_db.execute("SELECT * FROM face_info")
db_data = cursor.fetchall()

total_distances = []
total_names = []
for data in dn_data:
	total_names.append(data[1])
	db_embeddings = convert_array(data[2])
	distance = round(np.linalg.norm(db_embeddings-embeddings), 2)
	total_distances.append(distance)
total_result = dict(zip(total_names, total_distances))
idx_min = np.argmin(total_distances)

distance, name = total_distances[idx_min], total_names[idx_min]
conn_db.close()

if distance < threshold:
	return name, distance, total_result
else:
	name = "Unknown Person"
	return name, distance, total_result

接下来看看测试结果吧！由以下测试结果可以看出，在数据库中的人脸都有正确的识别到，而不在的则会显示Unkonwn Perrson。

详细代码详见https://github.com/chingi071/Face_recognition

参考目录

https://medium.com/ching-i/face-recognition-%E4%BA%BA%E8%87%89%E8%BE%A8%E8%AD%98-python-%E6%95%99%E5%AD%B8-75a5e2ef534f