视觉机器学习20讲-MATLAB源码示例（15）-BP学习算法

1. BP学习算法

BP算法就是目前使用较为广泛的一种参数学习算法.

BP(back propagation)神经网络是1986年由Rumelhart和McClelland为首的科学家提出的概念，是一种按照误差逆向传播算法训练的多层前馈神经网络。

BP算法是建立在BP神经网络（多层前馈网络）上的一种算法，它根据神经网络的路径进行一层一层的运算，这个运算包括信号的正向传播和误差的反向传播两部分。

2. Matlab仿真

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 功能:演示BP算法在计算机视觉中的应用
% 基于BP算法训练神经网络实现目标分类
%环境：Win7，Matlab2018a
% Modi:C.S
% 时间:2022-04-05
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% load training data
clear all;
data = load('mnist.mat');
train_x = double(data.train_x) / 255;
train_y = double(data.train_y);
test_x  = double(data.test_x) / 255;
test_y  = double(data.test_y);
clear('data');

%% NN train
input_dim           = size(train_x, 2);
output_dim          = size(train_y, 2);
hidden_sz           = 100;
nn.sizes            = [input_dim, hidden_sz, output_dim];
nn.n                = numel(nn.sizes);
nn.learning_rate    = 0.1;
nn.momentum         = 0.5;

opts.numepochs      = 2;
opts.batchsize      = 100;

rng('default');
for i = 1:nn.n - 1
    nn.W{i}  = (rand(nn.sizes(i+1), nn.sizes(i)+1) - 0.5) * 2 * 4 * sqrt(6 / (nn.sizes(i+1) + nn.sizes(i)));
    nn.vW{i} = zeros(size(nn.W{i}));
end
disp('Start to train NN using BP');
nn = nntrain(nn, train_x, train_y, opts);

%% NN test
disp('Start to test NN:');
[err_rate, err_num] = nntest(nn, test_x, test_y);
% With 2 epochs, the error rate: could be around 10%.
disp(['Final classification error rate: ' num2str(err_rate*100), '%.']);

3. 仿真结果

>> BPLearning
Start to train NN using BP
NN train: epoch 1/2. Took 1.9522 seconds. Mini-batch mean squared error on training set is 0.28579; Full-batch train err = 0.162176
NN train: epoch 2/2. Took 1.9844 seconds. Mini-batch mean squared error on training set is 0.13741; Full-batch train err = 0.121116
Start to test NN:
Final classification error rate: 11.84%.

4. 小结

BP算法易形成局部极小而得不到全局最优，训练次数多使得学习效率低，存在收敛速度慢等问题。

传统的BP算法改进主要有两类：

启发式算法：如附加动量法，自适应算法。

数值优化算法：如共轭梯度法、牛顿迭代法等。

BP学习算法在深度学习的神经网络中应用最多，也可以说是学习深度学习的基础算法之一，有兴趣的推荐去仔细查看全文《机器学习20讲》中第十五讲内容，深入学习了解，MATLAB仿真源码在分享的资源中已打包好，欢迎取用。