Different Between LSTM and LSTMCell Function

vision
1

Hello I am still confuse what is the different between function of LSTM and LSTMCell. I have read the documentation however I can not visualize it in my mind the different between 2 of them. Suppose I want to creating this network in the picture.

Suppose green cell is the LSTM cell and I want to make it with depth=3, seq_len=7, input_size=3. Red cell is input and blue cell is output. What function should I use for this and how to use it?, Suppose I just want the output of the last sequence can I just using out[:, -1, :] ?
-Thank you-

6 Likes
2

Hi. As far as I understand your figure, the red box below is the Cell.

That is the reason Cell return

    h_1 (batch, hidden_size): tensor containing the next hidden state for each element in the batch
    c_1 (batch, hidden_size): tensor containing the next cell state for each element in the batch
3

Yes as I mention before suppose that green is LSTM, red is input, and blue is output. How can you use function of LSTM and LSTMCell which available in Pytorch and what is their difference if we have visualization like in the figure.

4

LSTMCell is more flexible and you need less code with LSTM .

So with LSTMCell,

def forward(self, x):
        h = self.get_hidden() 
        for input in x:  
            h = self.rnn(input, h) # self.rnn = self.LSTMCell(input_size, hidden_size)

while with LSTM it is

def forward(self, x):
        h_0 = self.get_hidden()
        output, h = self.rnn(x, h_0) # self.rnn = self.LSTM(input_size, hidden_size)

output is the blue rectangles in your fig.

13 Likes
5

Be aware of using various RNNCells instead of RNN/GRU/LSTM models itself, since Cells don’t have cuDNN optimisation and thus don’t have any fused operations, etc.

16 Likes
6

analvikingur: Are you trying to say that LSTMCell will not work well on GPU, where the LSMT class will work fine with GPU?

7

RNN/LSTM/GRU is boosted if cudnn is available.

Maybe, torch.jit will be ease the difference someday.

8

Just FYI https://github.com/pytorch/benchmark/tree/master/rnns includes JIT boosted RNNs

1 Like
9

@moskomule is that mean LSTMCell and LSTM do the same task? no difference in performanc. right? do the same task? only difference is in code?

10

I think the expected answer is in this. [LSTM and LSTMCell ？]

1 Like
12

I made this example and hope it helps RNNs/LSTMCellvsLSTM.ipynb at main · CarlosJose126/RNNs · GitHub

2 Likes