【笔记】3.7循环神经网络

密集层->循环神经网络

• 语言模型：预测下一个词
• 使用MLP不能很好的处理序列信息

RNN and Gated RNN

• 
• Gated RNN(LSTM, GRU): finer control of information flow
  -
  代码实现

W_xh=nn.Parameter(torch.randn(num_inputs,num_hiddens)*0.01)
W_hh=nn.Parameter(torch.randn(num_hiddens,num_hiddens)*0.01)
b_h=nn.Parameter(torch.zeros(num_hiddens))

H=torch.zeros(num_hiddens)
outputs=[]

for X in inputs:
    H=torch.tanh(X @ W_xh+H @ W_hh +b_h)
    outputs.append(H)

Bi-RNN and Deep RNN


模型选择

Summary:

• MLP:stack dense layers with non-linear activations
• CNN: stack convolution activation and pooling layers to efficient extract spatial information
• RNN: stack recurrent layers to pass temporal information through hidden state