分析建模，日常问题整理（二十五）
2019.4.29~2019.6.13
-
- 简单实现Python人脸识别
face_recognition是世界上最简洁的人脸识别库，可用Python和命令行工具提取、识别、操作人脸。基于业内领先的C++开源库dlib中的深度学习模型，用Labeled Faces in the Wild人脸数据集进行测试，有高达99.38%的准确率。当然dlib也有相应的功能，比face_recognition稍微繁琐一点。
首先：
安装cmake,dlib，再安装face_recognition
安装face_recognition出现问题，主要原因是没有安装正确对应dlib版本。
安装dlib版本各种问题，建议直接找对应版本的whl。
dlib下载地址
然后face_recognition就可以安装了。
丰富的人脸识别案例
问题：ft2(freetype2总是安装失败！)
'''
首先加载各种包
'''
%pylab inline
import face_recognition
import cv2
import matplotlib.patches as patches
from IPython.display import clear_output
from matplotlib.pyplot import imshow
import matplotlib.pylab as plt
from IPython.display import display_html
import matplotlib
from PIL import Image
matplotlib.rcParams['font.sans-serif'] = ['SimHei']
matplotlib.rcParams['font.family']='sans-serif'
plt.rcParams['axes.unicode_minus']=False
warnings.filterwarnings('ignore')
import IPython.core.display as di;
di.display_html('', raw=True)
1）识别人脸并输出
# Load the jpg file into a numpy array
image = face_recognition.load_image_file("你要识别的图片.jpg")
# image是n*m的像素点集合，每个元素代表这个像素点的灰度值
# Find all the faces in the image using the default HOG-based model.
# This method is fairly accurate, but not as accurate as the CNN model and not GPU accelerated.
# See also: find_faces_in_picture_cnn.py
face_locations = face_recognition.face_locations(image)
'''
face_locations为所有脸部所在方框的位置，CSS中上边距离上框像素距离值，右边距离左框像素距离值，下边距离上框像素距离值，左边距离左框像素距离值。跟image的数组有关
'''
print("I found {} face(s) in this photograph.".format(len(face_locations)))
for face_location in face_locations:
# Print the location of each face in this image
top, right, bottom, left = face_location
print("A face is located at pixel location Top: {}, Left: {}, Bottom: {}, Right: {}".format(top, left, bottom, right))
# You can access the actual face itself like this:
face_image = image[top:bottom, left:right]
pil_image = Image.fromarray(face_image)
## 这里是根据location定位并输出方框的位置
pil_image.show()
2）人脸匹配（增加细节参数、核心算法说明）
# coding:utf-8
import face_recognition
#输入已知图片biden.jpg
known_image = face_recognition.load_image_file("你已知的那个人图片.jpg")
#输入待识别的图片unknown.jpg
unknown_image = face_recognition.load_image_file("你打算识别的那个人图片.jpg")
biden_encoding = face_recognition.face_encodings(known_image)[0]
unknown_encoding = face_recognition.face_encodings(unknown_image)[0]
results = face_recognition.compare_faces([biden_encoding], unknown_encoding) ## biden_encoding是多个学习的人脸？
#输出的results是一串Boolean值
print ('是否是同一个人',results)
3）摄像头实时标记是哪个人
import face_recognition
import cv2
video_capture = cv2.VideoCapture(0)
## 打开摄像头，参数为0则打开内置摄像头，参数为路径则打开相应视频
# Load a sample picture and learn how to recognize it.
obama_image = face_recognition.load_image_file("兔兔.jpg")
obama_face_encoding = face_recognition.face_encodings(obama_image)[0]
# Load a second sample picture and learn how to recognize it.
biden_image = face_recognition.load_image_file("猴猴.jpg")
biden_face_encoding = face_recognition.face_encodings(biden_image)[0]
# Create arrays of known face encodings and their names
known_face_encodings = [ obama_face_encoding, biden_face_encoding]
known_face_names = ["兔兔", "猴猴"]
# Initialize some variables
face_locations = []
face_encodings = []
face_names = []
process_this_frame = True
while True:
# Grab a single frame of video 按帧读取视频，返回是否读取正确和对应一帧图像的矩阵
ret, frame = video_capture.read()
# Resize frame of video to 1/4 size for faster face recognition processing
small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25) ## 将图片缩小为原来的(0,0)倍
# Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
# opencv的图像是BGR格式的，而我们需要是的RGB格式的，因此需要进行一个转换
rgb_small_frame = small_frame[:, :, ::-1]
# Only process every other frame of video to save time
if process_this_frame:
# Find all the faces and face encodings in the current frame of video
face_locations = face_recognition.face_locations(rgb_small_frame) # 对没帧的人脸定位
face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)
# 对截取的图像进行编码，encoding(里面怎么会有两个参数呢)
face_names = []
for face_encoding in face_encodings:
# See if the face is a match for the known face(s)
matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
name = "Unknown"
# If a match was found in known_face_encodings, just use the first one.
if True in matches:
first_match_index = matches.index(True)
name = known_face_names[first_match_index]
face_names.append(name)
process_this_frame = not process_this_frame
# Display the results 显示识别到的人物
for (top, right, bottom, left), name in zip(face_locations, face_names):
# Scale back up face locations since the frame we detected in was scaled to 1/4 size
top *= 4
right *= 4
bottom *= 4
left *= 4
# Draw a box around the face加上矩形框
cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
# Draw a label with a name below the face 加上识别到的人名字或标签
cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
font = cv2.FONT_HERSHEY_DUPLEX
cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)
# Display the resulting image
cv2.imshow('Video', frame)
# Hit 'q' on the keyboard to quit! 按q退出
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()
应用场景：线上审批贷款时身份信息核实；识别是否来贷过款，是否命中黑名单图库等；识别异常微表情，判断欺诈的可能性。
-
- Python的修饰符作用和使用
@func(x)
def test(f):
print( "before ...")
f()
print( "after ...")
@test
## 这里的test相当于调用应用test函数，并运行它。运行的时候他会将将下面这个函数作为输入来运行。函数中有f()，因此也会直接运行下面这个函数。输出结果。
def func():
print( "func was called")
-
- MD5、SM3加密
MD5消息摘要算法,可以产生出一个128位（16字节）的散列值，用于确保信息传输完整一致。
用于个人信息脱敏
import hashlib
m2 = hashlib.md5()
m3 = hashlib.md5()
m4 = hashlib.md5()
lst = []
for x,y,z in zip(tem_no['v1'],tem_no['v2'],tem_no['v3']):
m2.update(str(a).encode('utf-8'))
m3.update(y.encode('utf-8'))
m4.update(z.encode('utf-8'))
lst.append([m2.hexdigest(),m3.hexdigest(),m4.hexdigest()])
lst = pd.DataFrame(lst, columns =['v1','v2','v3'])
GmSSL是一个开源的加密包的python实现，支持SM2/SM3/SM4等国密(国家商用密码)算法、项目采用对商业应用友好的类BSD开源许可证，开源且可以用于闭源的商业应用。SM3返回64位数的密码
pip install gmssl
from gmssl import sm3, func
sm3.sm3_hash(func.bytes_to_list('1232141'.encode('utf-8')))
## 或者sm3.sm3_hash(func.bytes_to_list(b'1232141'))
SM2/SM4见github的test
一般加密之后不能解密（否则就起不到加密效果），所以要提前设定对应编号，方便匹配。
-
- Python直接连接数据库取数
import cx_Oracle
import pymysql
import os
# mysql数据库
def get_data_from_mysql(sql):
ip='rm-8vbgcy0w95xho6n7eo.mysql.zhangbei.rds.aliyuncs.com'
port=3306
user='username'
password='passwordname'
dbname='pcl'
connection=pymysql.connect(host=ip,port=port,user=user,password=password,db=dbname,charset='utf8mb4')
with connection.cursor() as cursor:
cursor.execute(sql)
df=cursor.fetchall()
return df
sql_common = """select * from ods.table'"""
df_temp = get_data_from_oracle(sql_common)
# oracle数据库
def get_data_from_oracle(sql):
os.environ['NLS_LANG'] = 'SIMPLIFIED CHINESE_CHINA.UTF8'
configs = {
'username': 'username',
'password': 'passwordname',
'host': '221.236.20.211',
'port': '15213',
'service_name': 'orcl'
}
username = configs['username']
password = configs['password']
addr = '{}:{}/{}'.format(configs['host'], configs['port'], configs['service_name'])
db = cx_Oracle.connect(username, password, addr)
cr = db.cursor()
cr.execute(sql)
rx = cr.description
df = cr.fetchall()
cr.close()
db.close()
dfs=pd.DataFrame(df,columns=[i[0].lower() for i in rx])
return dfs
sql_common = """select * from ods.table'"""
df_temp = get_data_from_oracle(sql_common)