In LSRMCell, the pytorch computes the following function:
\begin{array}{ll} \
i_t = \sigma(W_{ii} x_t + b_{ii} + W_{hi} h_{(t-1)} + b_{hi}) \
f_t = \sigma(W_{if} x_t + b_{if} + W_{hf} h_{(t-1)} + b_{hf}) \
g_t = \tanh(W_{ig} x_t + b_{ig} + W_{hg} h_{(t-1)} + b_{hg}) \
o_t = \sigma(W_{io} x_t + b_{io} + W_{ho} h_{(t-1)} + b_{ho}) \
c_t = f_t * c_{(t-1)} + i_t * g_t \
h_t = o_t * \tanh(c_t) \
\end{array}
(Sorry I don’t konw how to convert it to LaTeX formulas)
I wonder what the dimension of W in LSTMCell is. e.g. W_{ii}, W_{hi}. I guess the output dimension of Wx(or Wh) is Hidden_Size * 1. Am I right? I can’t prove it, can anyone help me?