tensorflow机器学习之利用CNN卷积神经网络进行面部表情识别的实例代码

本例通过 TensorFlow 构造卷积神经网络，做表情识别的测试。

输入数据可以从http://download.csdn.net/user/shinian1987上下载FER-2013 这个数据库， 一共有 35887 张人脸灰度图像的数据，每张分辨率48*48，已经做好了csv文件，每张图片的表情由0~6数字表示，分别代表(0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral)。下载数据后，解压，将文件fer2013.csv放入和程序同一级目录下的'data'文件夹中即可。如果感兴趣可以把每张图像用PIL或者cv2库还原保存在本地，如下所示。

为了计算方便，本例中用其中的 30000张图像做训练，5000张图像做测试集，我们建立一个3个convolution layer 以及 3个 pooling layer 和一个 FC layer 的CNN 来做训练（再添加一级FC应该会更好些）。

CNN网络结构如下所示： 
input -> conv 1 -> pool 1 -> conv 2 -> pool 2 -> conv 3 -> pool 3 -> fc 1 -> out 
input -> 48×48 
conv 1 -> filter size: 3×3, “SAME” padding, output: 48×48 
pool 1 -> filter size: 2×2, output: 24×24 
conv 2 -> filter size: 3×3, “SAME” padding output: 24×24 
pool 2 -> filter size: 2×2, output: 12×12 
conv 3 -> filter size: 3×3, “SAME” padding output: 12×12 
pool 3 -> filter size: 2×2, output: 6×6 
fc 1 -> hidden nodes: 200, output: 1×100 
out -> 1×2

剩下直接上代码，代码很简单，试验环境是python3.6， tensorflow1.2.0，程序已经调通，复制粘贴就可以跑。感谢万水千山先生无私分享宝贵知识，欢迎大家踩他，http://blog.csdn.net/matrix_space，他使用的是python2和老版本的tensorflow的搭建系统，有些过去有效的函数在新版本的tensorflow需要修改。

程序的算法总流程如下：

1. 先读csv数据

2. 将其中label和图像部分分别获取并保存，设置batch，方便训练网络时喂数据。

3. 构建CNN网络。 

4. 训练，跑起来就可以去睡了，设置了跑100轮，要很长很长时间，电脑比较老，CPU（i5）的四个线程几乎一直满负荷，8G内存占用约5G左右，我的显卡是A卡，GPU还不支持，估计使用N卡同时使用GPU会好很多。

5. 验证。经过验证，这个网络的效果还不是很理想，大约成功率在25.6左右，还是很低，但作为初学构建CNN网络，还是有用的，如果大家设计出了识别率更高的网络，欢迎指导。

# -*- coding: utf-8 -*-
"""
@author: sqh4587
"""
import string, os, sys
import numpy as np
import matplotlib.pyplot as plt
import scipy.io
import random
import tensorflow as tf
import pandas as pd

dir_name = 'data'
print('----------- no sub dir')
print('The folder path: ', dir_name)
files = os.listdir(dir_name)
for f in files:
    print(dir_name + os.sep + f)

file_path = dir_name + os.sep + files[0]
print(file_path)
data = pd.read_csv(file_path, dtype='a')
label = np.array(data['emotion'])
img_data = np.array(data['pixels'])
N_sample = label.size
print(N_sample)
# print label.size
Face_data = np.zeros((N_sample, 48 * 48))
Face_label = np.zeros((N_sample, 7), dtype=int)
temp = np.zeros((7), dtype= int)
for i in range(N_sample):
    x = img_data[i]
    x = np.fromstring(x, dtype=float, sep=' ')
    x_max = x.max()
    x = x / (x_max + 0.0001)

    Face_data[i] = x
    Face_label[i, int(label[i])] = 1
    if i <10:
        print('i: %d \t '%(i), Face_label[i])

train_num = 30000
test_num = 5000
train_x = Face_data[0:train_num, :]
train_y = Face_label[0:train_num, :]
test_x = Face_data[train_num: train_num + test_num, :]
test_y = Face_label[train_num: train_num + test_num, :]
print("All is well")
batch_size = 50
train_batch_num = train_num / batch_size
test_batch_num = test_num / batch_size
train_epoch = 100
learning_rate = 0.001
# Network Parameters
n_input = 2304  # data input (img shape: 48*48)
n_classes = 7  # total classes
dropout = 0.5  # Dropout, probability to keep units
# tf Graph input
x = tf.placeholder(tf.float32, [None, n_input])
y = tf.placeholder(tf.float32, [None, n_classes])
keep_prob = tf.placeholder(tf.float32)  # dropout (keep probability)


# Create some wrappers for simplicity
def conv2d(x, W, b, strides=1):
    # Conv2D wrapper, with bias and relu activation
    x = tf.nn.conv2d(x, W, strides=[1, strides, strides, 1], padding='SAME')
    x = tf.nn.bias_add(x, b)
    return tf.nn.relu(x)


def maxpool2d(x, k=2):
    # MaxPool2D wrapper
    return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, k, k, 1],
                          padding='VALID')


# Create model
def conv_net(x, weights, biases, dropout):
    # Reshape input picture
    x = tf.reshape(x, shape=[-1, 48, 48, 1])
    # Convolution Layer
    conv1 = conv2d(x, weights['wc1'], biases['bc1'])
    # Max Pooling (down-sampling)
    conv1 = maxpool2d(conv1, k=2)
    # Convolution Layer
    conv2 = conv2d(conv1, weights['wc2'], biases['bc2'])
    # Max Pooling (down-sampling)
    conv2 = maxpool2d(conv2, k=2)
    # Convolution Layer
    conv3 = conv2d(conv2, weights['wc3'], biases['bc3'])
    # Max Pooling (down-sampling)
    conv3 = maxpool2d(conv3, k=2)
    # Fully connected layer
    # Reshape conv2 output to fit fully connected layer input
    fc1 = tf.reshape(conv3, [-1, weights['wd1'].get_shape().as_list()[0]])
    fc1 = tf.add(tf.matmul(fc1, weights['wd1']), biases['bd1'])
    fc1 = tf.nn.relu(fc1)
    # Apply Dropout
    fc1 = tf.nn.dropout(fc1, dropout)
    # Output, class prediction
    out = tf.add(tf.matmul(fc1, weights['out']), biases['out'])
    return out


# Store layers weight & bias
weights = {
    # 3x3 conv, 1 input, 128 outputs
    'wc1': tf.Variable(tf.random_normal([3, 3, 1, 128])),
    # 3x3 conv, 128 inputs, 64 outputs
    'wc2': tf.Variable(tf.random_normal([3, 3, 128, 64])),
    # 3x3 conv, 64 inputs, 32 outputs
    'wc3': tf.Variable(tf.random_normal([3, 3, 64, 32])),
    # fully connected,
    'wd1': tf.Variable(tf.random_normal([6 * 6 * 32, 200])),
    # 1024 inputs, 10 outputs (class prediction)
    'out': tf.Variable(tf.random_normal([200, n_classes]))
}

biases = {
    'bc1': tf.Variable(tf.random_normal([128])),
    'bc2': tf.Variable(tf.random_normal([64])),
    'bc3': tf.Variable(tf.random_normal([32])),
    'bd1': tf.Variable(tf.random_normal([200])),
    'out': tf.Variable(tf.random_normal([n_classes]))
}
# Construct model
pred = conv_net(x, weights, biases, keep_prob)
# Define loss and optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
# Evaluate model
correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
# Initializing the variables
init = tf.initialize_all_variables()
Train_ind = np.arange(train_num)
Test_ind = np.arange(test_num)
with tf.Session() as sess:
    sess.run(init)
    for epoch in range(0, train_epoch):
        Total_test_loss = 0
        Total_test_acc = 0
        for train_batch in range(0, int(train_batch_num)):
            sample_ind = Train_ind[train_batch * batch_size:(train_batch + 1) * batch_size]
            batch_x = train_x[sample_ind, :]
            batch_y = train_y[sample_ind, :]
            # Run optimization op (backprop)
            sess.run(optimizer, feed_dict={x: batch_x, y: batch_y,
                                           keep_prob: dropout})
            if train_batch % batch_size == 0:
                # Calculate loss and accuracy
                loss, acc = sess.run([cost, accuracy], feed_dict={x: batch_x,
                                                                  y: batch_y,
                                                                  keep_prob: 1.})
                print("Epoch: " + str(epoch + 1) + ", Batch: " + str(train_batch) + ", Loss= " + \
                      "{:.3f}".format(loss) + ", Training Accuracy= " + \
                      "{:.3f}".format(acc))
        # Calculate test loss and test accuracy
        for test_batch in range(0, int(test_batch_num)):
            sample_ind = Test_ind[test_batch * batch_size:(test_batch + 1) * batch_size]
            batch_x = test_x[sample_ind, :]
            batch_y = test_y[sample_ind, :]
            test_loss, test_acc = sess.run([cost, accuracy], feed_dict={x: batch_x,
                                                                        y: batch_y,
                                                                        keep_prob: 1.})
            Total_test_lost = Total_test_loss + test_loss
            Total_test_acc = Total_test_acc + test_acc

        Total_test_acc = Total_test_acc / test_batch_num
        Total_test_loss = Total_test_lost / test_batch_num
        print("Epoch: " + str(epoch + 1) + ", Test Loss= " + \
              "{:.3f}".format(Total_test_loss) + ", Test Accuracy= " + \
              "{:.3f}".format(Total_test_acc))
plt.subplot(2, 1, 1)
plt.ylabel('Test loss')
plt.plot(Total_test_loss, 'r')
plt.subplot(2, 1, 2)
plt.ylabel('Test Accuracy')
plt.plot(Total_test_acc, 'r')

print("All is well")
plt.show()