RNNCell使用

标签:abi   over   down   imp   cli   header   https   rev   build   

目录
  • Recap
  • input dim, hidden dim
  • SimpleRNNCell
    • Single layer RNN Cell
  • Multi-Layers RNN
    • RNN Layer

Recap

43-RNNCell使用-rnn.jpg

input dim, hidden dim

from tensorflow.keras import layers

# \(xw_{xh} + hw_{nn}\),3次
cell = layers.SimpleRNNCell(3)
cell.build(input_shape=(None, 4))

cell.trainable_variables

[<tf.Variable ‘kernel:0‘ shape=(4, 3) dtype=float32, numpy=
 array([[-0.5311725 ,  0.7757399 , -0.19041312],
        [ 0.90420175, -0.14276218,  0.1546886 ],
        [ 0.81770146, -0.46731013, -0.05373603],
        [ 0.49086082,  0.10275221,  0.10146773]], dtype=float32)>,
 <tf.Variable ‘recurrent_kernel:0‘ shape=(3, 3) dtype=float32, numpy=
 array([[ 0.7557267 , -0.58395827,  0.2964283 ],
        [-0.64145935, -0.56886935,  0.5147014 ],
        [-0.13193521, -0.5791204 , -0.8044953 ]], dtype=float32)>,
 <tf.Variable ‘bias:0‘ shape=(3,) dtype=float32, numpy=array([0., 0., 0.], dtype=float32)>]

SimpleRNNCell



  • o


    u


    t


    ,



    h


    1



    =


    c


    a


    l


    l


    (


    x


    ,



    h


    0



    )

    • x: [b,seq len,word vec]




    • h


      0




      /




      h


      1



      :


      [


      b


      ,


      h


      d


      i


      m


      ]

    • out: [b,h dim]

Single layer RNN Cell

import tensorflow as tf

x = tf.random.normal([4, 80, 100])
ht0 = x[:, 0, :]

cell = tf.keras.layers.SimpleRNNCell(64)

out, ht1 = cell(ht0, [tf.zeros([4, 64])])

out.shape, ht1[0].shape

[]





(TensorShape([4, 64]), TensorShape([4, 64]))
id(out), id(ht1[0])  # same id
(4877125168, 4877125168)

Multi-Layers RNN

43-RNNCell使用-多层rnn.jpg

x = tf.random.normal([4, 80, 100])
ht0 = x[:, 0, :]

cell = tf.keras.layers.SimpleRNNCell(64)
cell2 = tf.keras.layers.SimpleRNNCell(64)
state0 = [tf.zeros([4, 64])]
state1 = [tf.zeros([4, 64])]

out0, state0 = cell(ht0, state0)
out2, state2 = cell2(out, state2)

out2.shape, state2[0].shape

(TensorShape([4, 64]), TensorShape([4, 64]))

RNN Layer

self.run = keras.Sequential([
    layers.SimpleRNN(units,dropout=0.5,return_sequences=Ture,unroll=True),
    layers.SimpleRNN(units,dropout=0.5,unroll=True)
])
x=self.rnn(x)

RNNCell使用

标签:abi   over   down   imp   cli   header   https   rev   build   

原文地址:https://www.cnblogs.com/abdm-989/p/14123414.html

版权声明:完美者 发表于 2020-12-17 12:24:16。
转载请注明:RNNCell使用 | 完美导航

暂无评论

暂无评论...