机器学习之人脸识别face_recognition使用

简介

一

face_recognition项目是世界上最简洁的人脸识别库，可以使用Python和命令行工具提取、识别、操作人脸。
本项目的人脸识别是基于业内领先的C++开源库 dlib中的深度学习模型，用Labeled Faces in the Wild人脸数据集进行测试，有高达99.38%的准确率。但对小孩和亚洲人脸的识别准确率尚待提升。
Labeled Faces in the Wild是美国麻省大学安姆斯特分校（University of Massachusetts Amherst)制作的人脸数据集，该数据集包含了从网络收集的13,000多张面部图像。
本项目提供了简易的face_recognition命令行工具

二

人脸识别实际上是对人脸进行编码后再去计算两两人脸的相似度，known_image是已知人脸库的图像，unknown_image是待检测的图像，分别利用face_encodings函数来映射成一个向量，再利用两个向量的内积来衡量相似度，compare_faces函数就是根据阈值确认是否是同一人脸。上述函数都是支持多个人脸计算的。另外compare_faces有个tolerance参数是控制阈值的，tolerance值越低越严格，默认为0.6。

主要方法介绍

1. load_image_file 加载图像

import face_recognition之后直接调用face_recognition.load_image_file()读入图像，参数给文件名字符串，注意：jpg文件要和程序放在同一个文件夹下。输出图像是rgb格式（opencv中是bgr格式）。

import face_recognition
#加载图像文件
image = face_recognition.load_image_file("your_file.jpg")

2. face_locations 定位图中所有人脸

加载图像文件后直接调用face_recognition.face_locations(image)，能定位所有图像中识别出的人脸位置信息，返回值是列表形式，列表中每一行是一张人脸的位置信息，包括[top, right, bottom, left]，也可理解为每个人脸是一个tuple存储，分别代表框住人脸的矩形中左上角和右下角的坐标（x1,y1,x2,y2）。可遍历列表打印出每张脸的位置信息，也可以通过位置信息截出识别出的人脸的图像显示出来，代码如下：

#定位所有找到的脸的位置
face_locations = face_recognition.face_locations(image)
# 循环找到的所有人脸
for face_location in face_locations:
        # 打印每张脸的位置信息
        top, right, bottom, left = face_location
        print("A face is located at pixel location Top: {}, Left: {}, Bottom: {}, Right: {}".format(top, left, bottom, right))
        # 指定人脸的位置信息，然后显示人脸图片
        face_image = image[top:bottom, left:right]
        pil_image = Image.fromarray(face_image)
        pil_image.show()

3. face_landmarks 识别人脸关键点

加载图像后，调用face_recognition.face_landmarks(image)可识别出人脸关键点信息，包括眼睛、鼻子、嘴巴和下巴等，参数仍是加载的图像image，返回值是包含面部特征字典的列表，列表中每一项对应一张人脸，包括nose_bridge、right_eyebrow、right_eye、chine、left_eyebrow、bottom_lip、nose_tip、top_lip、left_eye几个部分，每个部分包含若干个特征点(x,y)，总共有68个特征点。列表长度就是图中识别出的人脸数，可遍历此列表和字典的键值对，打印出所有面部特征点，也可在图像上画出来，代码如下：

from PIL import Image, ImageDraw
face_landmarks_list = face_recognition.face_landmarks(image)
pil_image = Image.fromarray(image)
d = ImageDraw.Draw(pil_image)
for face_landmarks in face_landmarks_list:
    #face_landmarks_list中每个元素都包含以下key的字典
    #打印此图像中每个面部特征的位置
    facial_features = [
        'chin',
        'left_eyebrow',
        'right_eyebrow',
        'nose_bridge',
        'nose_tip',
        'left_eye',
        'right_eye',
        'top_lip',
        'bottom_lip'
    ]
    for facial_feature in facial_features:
        print("The {} in this face has the following points: {}".format(facial_feature, face_landmarks[facial_feature]))
    #在图像中画出每个人脸特征！
    for facial_feature in facial_features:
        d.line(face_landmarks[facial_feature], width=5)
pil_image.show()

4. face_encodings 获取图像文件中所有面部编码

face_recognition.face_encodings(image)方法可获取每个图像文件中每个面部的面部编码,参数仍是加载的图像image，由于每个图像中可能有多个脸，所以返回的是一个编码列表，后续访问时注意加上索引号或者依次遍历。
由打印可看出每张人脸是一个128维的向量。

face_encodings = face_recognition.face_encodings(image)
for face_encoding in face_encodings:
    print("face_encoding len = {} \nencoding:{}\n\n".format(len(face_encoding),face_encoding))

5. compare_faces 由面部编码匹配脸

用face_recognition.compare_faces()方法可匹配两个面部特征编码，利用两个向量的内积来衡量相似度，根据阈值确认是否是同一人脸。
位置参数：第一个参数给出一个面部编码列表（很多张脸），第二个参数给出单个面部编码（一张脸），compare_faces方法会将第二个参数的编码与第一个参数中的编码依次匹配，返回值是一个布尔值列表，匹配成功的脸返回True，匹配失败的返回False，顺序与第一个参数中脸的顺序一致。
默认参数：tolerance=0.6,可根据自己需求更改，tolerance越小匹配越严格，例如
matches = face_recognition.compare_faces(known_face_encodings, face_encoding,tolerance=0.39)，将阈值改为了0.39,阈值太低容易造成无法成功识别人脸，太高容易造成人脸识别混淆，这个是根据我自己测试后慢慢试出来的一个值。

matche_result = face_recognition.compare_faces(known_faces, face_encoding[0],tolerance=0.39)[0]

实践

demo1：识别人脸特征并打印出来

from PIL import Image, ImageDraw
import face_recognition

# 将jpg文件加载到numpy 数组中
image = face_recognition.load_image_file("view_Mr_G.jpg")

#查找图像中所有面部的所有面部特征
face_landmarks_list = face_recognition.face_landmarks(image)

print("I found {} face(s) in this photograph.".format(len(face_landmarks_list)))
pil_image = Image.fromarray(image)
d = ImageDraw.Draw(pil_image)
for face_landmarks in face_landmarks_list:
    #打印此图像中每个面部特征的位置
    facial_features = [
        'chin',
        'left_eyebrow',
        'right_eyebrow',
        'nose_bridge',
        'nose_tip',
        'left_eye',
        'right_eye',
        'top_lip',
        'bottom_lip'
    ]
    for facial_feature in facial_features:
        print("The {} in this face has the following points: {}".format(facial_feature, face_landmarks[facial_feature]))
    #在图像中画出每个人脸特征！
    for facial_feature in facial_features:
        d.line(face_landmarks[facial_feature], width=5)

pil_image.show()

demo2 : 识别图片中的所有人脸并显示

from PIL import Image
import face_recognition

# 将jpg文件加载到numpy 数组中
image = face_recognition.load_image_file("view_Mr_G.jpg")

# 使用默认的给予HOG模型查找图像中所有人脸
# 这个方法已经相当准确了，但还是不如CNN模型那么准确，因为没有使用GPU加速
# 另请参见: find_faces_in_picture_cnn.py
face_locations = face_recognition.face_locations(image)

# 使用CNN模型
# face_locations = face_recognition.face_locations(image, number_of_times_to_upsample=0, model="cnn")

# 打印：我从图片中找到了 多少 张人脸
print("I found {} face(s) in this photograph.".format(len(face_locations)))

# 循环找到的所有人脸
for face_location in face_locations:

        # 打印每张脸的位置信息
        top, right, bottom, left = face_location
        print("A face is located at pixel location Top: {}, Left: {}, Bottom: {}, Right: {}".format(top, left, bottom, right))
# 指定人脸的位置信息，然后显示人脸图片
        face_image = image[top:bottom, left:right]
        pil_image = Image.fromarray(face_image)
        pil_image.show()

demo3 : 显示未知图片中已知人物的脸

# 识别人脸鉴定是哪个人

import face_recognition
from PIL import Image, ImageDraw
import cv2
PATH = "/Users/tanyashi/Python/python project/face_recognition/一起同过窗"
VIEW_PIC_NAME="view2.jpg"
UNKNOWN_IMAGE="view_Mr_G.jpg"
#将jpg文件加载到numpy数组中
view_image = face_recognition.load_image_file(VIEW_PIC_NAME)
#要识别的图片
unknown_image = face_recognition.load_image_file(UNKNOWN_IMAGE)
results = []
#获取每个图像文件中每个面部的面部编码
#由于每个图像中可能有多个面，所以返回一个编码列表。
#但是由于我知道每个图像只有一个脸，我只关心每个图像中的第一个编码，所以我取索引0。
view_face_encoding = face_recognition.face_encodings(view_image)[0]
print("view_face_encoding:{}\n\n".format(view_face_encoding))
known_faces = [view_face_encoding]
face_locations = face_recognition.face_locations(unknown_image)
print('got {} face(s) in {}:'.format(len(face_locations), UNKNOWN_IMAGE))
for face_location in face_locations:
    top, right, bottom, left = face_location
    #print(top, right, bottom, left)
    face_image = unknown_image[top:bottom, left:right]
    face_encoding = face_recognition.face_encodings(face_image)
    if face_encoding:
        result = {}
        result['face_encoding'] = face_encoding
        result['is_view'] = face_recognition.compare_faces(known_faces, face_encoding[0])[0]
        result['location'] = face_location
        result['face_id'] = face_locations.index(face_location)
        results.append(result)
        if result['is_view']:
            print('face {} is view!!'.format(result['face_id']+1))
            #print(top, right, bottom, left)
#print("results :{}".format([i['is_view'] for i in results]))
print("please find out view in this image:")
view_face_location = [i['location'] for i in results if i['is_view']]
if view_face_location:
    top, right, bottom, left = view_face_location[0]
    #print(top, right, bottom, left)
    face_image = unknown_image[top:bottom, left:right]
    pil_image = Image.fromarray(face_image)
    pil_image.show()
else:
    print('view is not in this pic!')

demo4 : 摄像头头像识别

# 摄像头头像识别
import face_recognition
import cv2

video_capture = cv2.VideoCapture(0)

# 本地图像
chenduling_image = face_recognition.load_image_file("chenduling.jpg")
chenduling_face_encoding = face_recognition.face_encodings(chenduling_image)[0]

# 本地图像二
sunyizheng_image = face_recognition.load_image_file("sunyizheng.jpg")
sunyizheng_face_encoding = face_recognition.face_encodings(sunyizheng_image)[0]

# 本地图片三
zhangzetian_image = face_recognition.load_image_file("zhangzetian.jpg")
zhangzetian_face_encoding = face_recognition.face_encodings(zhangzetian_image)[0]

# Create arrays of known face encodings and their names
# 脸部特征数据的集合
known_face_encodings = [
    chenduling_face_encoding,
    sunyizheng_face_encoding,
    zhangzetian_face_encoding
]

# 人物名称的集合
known_face_names = [
    "michong",
    "sunyizheng",
    "chenduling"
]

face_locations = []
face_encodings = []
face_names = []
process_this_frame = True

while True:
    # 读取摄像头画面
    ret, frame = video_capture.read()

    # 改变摄像头图像的大小，图像小，所做的计算就少
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)

    # opencv的图像是BGR格式的，而我们需要是的RGB格式的，因此需要进行一个转换。
    rgb_small_frame = small_frame[:, :, ::-1]

    # Only process every other frame of video to save time
    if process_this_frame:
        # 根据encoding来判断是不是同一个人，是就输出true，不是为flase
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # 默认为unknown
            matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
            name = "Unknown"

            # if match[0]:
            #     name = "michong"
            # If a match was found in known_face_encodings, just use the first one.
            if True in matches:
                first_match_index = matches.index(True)
                name = known_face_names[first_match_index]
            face_names.append(name)

    process_this_frame = not process_this_frame

    # 将捕捉到的人脸显示出来
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4

        # 矩形框
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

        #加上标签
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    # Display
    cv2.imshow('monitor', frame)

    # 按Q退出
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

video_capture.release()
cv2.destroyAllWindows()

demo5人脸识别签到系统

# import face_recognition
import face_recognition
import cv2
import os
import numpy as np
from PIL import Image, ImageDraw, ImageFont
import dlib
import datetime
import threading  # 导入threading模块
import time
from imutils import face_utils
from scipy.spatial import distance

import yagmail


class Recorder:
    pass


record_dic = {}  # 访问记录字典
unknownjpg = []  # 保存来访的陌生人的照片的列表
flagover = 0  # 用于全局判断是否需要保存来访信息


def sendemail(title, contents, fileslist):
    yag = yagmail.SMTP(user="[email protected]", password="agjnddqxmibydjej", host='smtp.qq.com')
    yag.send('[email protected]', title, contents, fileslist)


# 双眼上下眼距检测
def eye_aspect_ratio(eye):
    # print(eye)
    A = distance.euclidean(eye[1], eye[5])  # euclidean是计算欧氏距离
    B = distance.euclidean(eye[2], eye[4])
    C = distance.euclidean(eye[0], eye[3])
    ear = (A + B) / (2.0 * C)
    return ear


# 字典转换为字符串
def dicttostr():
    strlist = []
    listkey = list(sorted(record_dic.keys()))
    for item in listkey:
        strlist.extend([item + ',' + str(onetime) for onetime in record_dic[item].times])
    return strlist


def saveRecorder(name, frame):
    global record_dic
    global flagover
    global unknownjpg
    if flagover == 1:
        return
    try:
        rec = record_dic[name]
        secondsDiff = (datetime.datetime.now() - rec.times[-1]).total_seconds()

        if secondsDiff < 60 * 10:  # 如果两次比较的时间在10分钟内将被过滤掉
            return
        rec.times.append(datetime.datetime.now())
        print('更新记录', record_dic, rec.times)
    except (KeyError):
        newRec = Recorder()
        newRec.times = [datetime.datetime.now()]
        record_dic[name] = newRec
        print('添加记录', record_dic, newRec.times)

    if name == '未知人员':
        s = str(record_dic[name].times[-1])
        print('写入', s[:10] + s[-6:])
        filename = s[:10] + s[-6:] + '.jpg'
        cv2.imwrite(filename, frame)
        unknownjpg.append(filename)


def loop_timer_handler():  # 定时器循环触发函数
    print('————————Timer headle!————————', str(datetime.datetime.now()))
    global timer2
    global flagover
    global record_dic
    global unknownjpg
    flagover = 1
    timer2 = threading.Timer(60 * 1, loop_timer_handler)  # 创建定时器 每1分钟执行一次
    timer2.start()

    # 如果mail_content不为空，则发送邮件通知
    mail_content = '\n'.join(dicttostr())
    if mail_content.strip():  # 如果有新的记录内容就发送邮件
        sendemail("来访统计记录", mail_content, unknownjpg)
        print('来访登记记录邮件已发送', mail_content)

    record_dic.clear()
    unknownjpg.clear()
    print("清空")

    time.sleep(10)
    print("重新开始")
    flagover = 0


timer2 = threading.Timer(2, loop_timer_handler)
timer2.start()


def load_img(sample_dir):
    print('加载已知人员图片...')

    for (dirpath, dirnames, filenames) in os.walk(sample_dir):  # 一级一级的文件夹递归
        print(dirpath, dirnames, filenames)
        facelib = []
        for filename in filenames:
            filename_path = os.sep.join([dirpath, filename])
            faceimage = face_recognition.load_image_file(filename_path)
            # 但是由于我知道每个图像只有一个脸，我只关心每个图像中的第一个编码，所以我取索引0
            face_encoding = face_recognition.face_encodings(faceimage)[0]
            facelib.append(face_encoding)
        return facelib, filenames


def add_text_cv2(text, left, bottom):
    '''
    由于cv2不支持中文显示，所以在视频流上显示中文时，需要先将帧数据转换成PIL图像，然后利用PIL来添加中文
    :param text: 要在图像上添加的文字 
    :param left: 添加文字离左边框的距离
    :param bottom: 添加文字离下边框距离
    :return: 
    '''
    global frame
    img_PIL = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))  # 转换图片格式
    font = ImageFont.truetype('simhei.ttf', 40)  # 加载字体
    position = (left, bottom)  # 指定文字输出位置
    draw = ImageDraw.Draw(img_PIL)  # 绘制照片
    draw.text(position, text, font=font, fill=(255, 255, 255))  # 绘制文字
    frame = cv2.cvtColor(np.asarray(img_PIL), cv2.COLOR_RGB2BGR)  # 将图片转回OpenCV格式


facelib, facename = load_img('../facelib')

video_capture = cv2.VideoCapture(0)  # 获得摄像头

face_locations = []  # 定义列表存放人脸位置
face_encodings = []  # 定义列表存放人脸特征编码
process_this_frame = True  # 定义信号量

frame_counter = 0  # 连续帧计数
EYE_AR_THRESH = 0.3  # EAR阈值
EYE_AR_CONSEC_FRAMES = 3  # 当EAR小于阈值时，接连多少帧一定发生眨眼动作

# 对应特征点的序号
RIGHT_EYE_START = 37 - 1
RIGHT_EYE_END = 42 - 1
LEFT_EYE_START = 43 - 1
LEFT_EYE_END = 48 - 1

while True:
    ret, frame = video_capture.read()  # 捕获一帧图片

    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)  # 将图片缩小1/4，为人脸识别提速
    # opencv的图像格式默认是BGR格式，大家可以想象成像素排列跟RGB格式不一样，所以我们必须做一点调整，将像素点进行反向
    rgb_small_frame = small_frame[:, :, ::-1]  # 将opencv的BGR格式转为RGB格式
    # 找到这一帧图像上所有的人脸位置
    face_locations = face_recognition.face_locations(rgb_small_frame)
    # 得到面部编码（见资料文章）
    face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)
    # print('人脸特征编码长度', len(face_encodings))
    face_names = []  # 定义列表，放置识别出来的人员的名字
    for face_encoding in face_encodings:  # 循环多张人脸， 这里一般只有一张人脸
        matches = face_recognition.compare_faces(facelib, face_encoding, tolerance=0.39)
        name = "未知人员"  # 定义默认的识别结果为Unknown
        # 有true说明图片对比已经成功，取出识别的人员名字并开始进行活体眨眼检测
        if True in matches:  # 如果识别出来，就将名称取出

            # 这里matches是个列表，包含每张已知图片的对比结果，这里应该只有一个为true
            # 所以我们要找出true的位置，这个位置就是找到的那个人的图片
            first_match_index = matches.index(True)
            # 取出这个人的名字
            name = facename[first_match_index][:-4]  # -4是去掉文件后缀
            print('开始活体检测...')
            # 在图上加上请眨眼的提示
            add_text_cv2('{}请眨眼...'.format(name), 0, 0)  # 在图像上添加眨眼提示

            detector = dlib.get_frontal_face_detector()  # 人脸检测器
            # 特征检测算法要做个介绍
            predictor = dlib.shape_predictor('../libs/shape_predictor_68_face_landmarks.dat')  # 人脸特征点检测器

            gray = cv2.cvtColor(rgb_small_frame, cv2.COLOR_BGR2GRAY)

            # 人脸检测，1代表把图片像素放大1倍以便能够搜集到更多的照片细节，返回检测到所有人脸区域的数组
            # 这个数不能设置过大，否则影响运行速度
            rects = detector(gray, 1)

            if len(rects) > 0:
                print('#' * 20)
                shape = predictor(gray, rects[0])  # 检测特征点，rect[0]代表取第一个人脸
                # 将所有的脸部坐标点转换为numpy array的格式
                # convert the facial landmark (x, y)-coordinates to a NumPy array
                points = face_utils.shape_to_np(shape)
                leftEye = points[LEFT_EYE_START:LEFT_EYE_END + 1]  # 取出左眼对应的特征点
                rightEye = points[RIGHT_EYE_START:RIGHT_EYE_END + 1]  # 取出右眼对应的特征点
                leftEAR = eye_aspect_ratio(leftEye)  # 计算左眼EAR
                rightEAR = eye_aspect_ratio(rightEye)  # 计算右眼EAR

                ear = (leftEAR + rightEAR) / 2.0  # 求左右眼EAR的均值

                # 如果EAR小于阈值，开始计算连续帧，只有连续帧计数超过EYE_AR_CONSEC_FRAMES时，才会计做一次眨眼
                if ear < EYE_AR_THRESH:
                    frame_counter += 1
                    print('ear小于阈值了！！！', frame_counter)
                if frame_counter >= EYE_AR_CONSEC_FRAMES:
                    # blink_counter += 1
                    print('眨眼检测成功，请进入！')
                    frame_counter = 0

                print('活体检测结束')

        face_names.append(name)  # 保存识别结果

    # process_this_frame = not process_this_frame	#信号量保护结束(这个语句没什么用)

    # 显示结果
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        top *= 4  # 还原人脸的原始尺寸，原来是1/4，现在放到原大小
        right *= 4
        bottom *= 4
        left *= 4
        # frame：要显示的帧，(left, top), (right, bottom) 各方向坐标，(0, 0, 255) 人脸框颜色， 2是线条粗细
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)  # 标注人脸
        add_text_cv2(name, left + 6, bottom - 6)
        saveRecorder(name, frame)  # 过滤并保存记录
    cv2.imshow('security check', frame)  # 显示图片检测框

    if cv2.waitKey(1) & 0xFF == ord('q'):  # 等待键盘输入，当输入q时，整个程序退出
        break

video_capture.release()  # 释放摄像头资源
time.sleep(2)  # 休眠10秒
timer2.cancel()  # 关闭定时器线程

谢谢大家，如有需要，请联系我

邮箱为[email protected]