DNNs & Papers in kaldi

出处：http://kaldi-asr.org/doc/dnn.html

Karel's DNN implementation

DNN: egs/wsj/s5/local/nnet/run_dnn.sh

• steps/nnet/make_fmllr_feats.sh
  storing 40-dimensional fMLLR features to disk
• steps/nnet/pretrain_dbn.sh
  预训练一个深度信念网络(Deep Belief Network，由多个限制玻尔兹曼机Restricted Boltzmann Machines层组成)。
  论文 Geoff Hinton's tutorial paper
• steps/nnet/train.sh
  frame cross-entropy training，该阶段训练一个能将帧分类为三音素状态的DNN
  论文 Deep Neural Networks for Acoustic Modeling in Speech Recognition
• steps/nnet/train_mpe.sh
  sMBR sequence-discriminative training，该阶段联合优化所有的句子来训练整个神经网络，比帧级的训练更接近ASR的目标。
  论文 Sequence-discriminative training of deep neural networks

CNN: egs/rm/s5/local/nnet/run_cnn.sh
论文 Deep Convolutional Neural Networks for LVCSR

Autoencoder training: egs/timit/s5/local/nnet/run_autoencoder.sh

Tandem system: egs/swbd/s5c/local/nnet/run_dnn_tandem_uc.sh
uc = Universal context network
论文 Convolutive Bottleneck Network Features for LVCSR

Dan's DNN implementation

• egs/wsj/s5/steps/nnet2/train_pnorm_fast.sh
  this is generally a p-norm network
  论文 Improving Deep Neural Network Acoustic Models Using Generalized Maxout Networks
• nnet3
  支持更多种类的网络（如LSTM），而不只是简单的前馈网络。
  论文 Parallel training of DNNs with Natural Gradient and Parameter Averaging