目录
- Recap
- input dim, hidden dim
- SimpleRNNCell
- Single layer RNN Cell
- Multi-Layers RNN
- RNN Layer
Recap
input dim, hidden dim
from tensorflow.keras import layers
# $xw_{xh} + hw_{nn}$，3次
cell = layers.SimpleRNNCell(3)
cell.build(input_shape=(None, 4))
cell.trainable_variables[,
,
] SimpleRNNCell
- \(out,h_1 = call(x,h_0)\)
x: [b,seq len,word vec]
\(h_0/h_1: [b,h dim]\)
out: [b,h dim]
Single layer RNN Cell
import tensorflow as tf
x = tf.random.normal([4, 80, 100])
ht0 = x[:, 0, :]
cell = tf.keras.layers.SimpleRNNCell(64)
out, ht1 = cell(ht0, [tf.zeros([4, 64])])
out.shape, ht1[0].shape[]
(TensorShape([4, 64]), TensorShape([4, 64]))id(out), id(ht1[0]) # same id(4877125168, 4877125168)Multi-Layers RNN
x = tf.random.normal([4, 80, 100])
ht0 = x[:, 0, :]
cell = tf.keras.layers.SimpleRNNCell(64)
cell2 = tf.keras.layers.SimpleRNNCell(64)
state0 = [tf.zeros([4, 64])]
state1 = [tf.zeros([4, 64])]
out0, state0 = cell(ht0, state0)
out2, state2 = cell2(out, state2)
out2.shape, state2[0].shape(TensorShape([4, 64]), TensorShape([4, 64]))RNN Layer
self.run = keras.Sequential([
layers.SimpleRNN(units,dropout=0.5,return_sequences=Ture,unroll=True),
layers.SimpleRNN(units,dropout=0.5,unroll=True)
])
x=self.rnn(x)