RNN-李宏毅

架构

基本形式

变形

Menory

LSTM（long short-term Menory）

性质

• taining rnn一般用 sigmod而不用relu.

解决办法

• LSTM（可以handle gradient vanishing 问题）：the influencr never disappears unless forget gat is closed.no gradient vansihing only if forget gate is opened.
• Clockwise RNN
• structurally Constrained Recurrent Network （SCRN）

many to one

• sentiment analysis
• Key terms

many to many

• speech recognition， connectionist temporal alassification(CTC):解决重叠字的问题
• sequence to sequence
• beyond sequence