Can the input of LSTM do not need hidden state?

Young19 · May 31, 2021, 1:19pm

I found that the input of the LSTM in the figure does not have a hidden state value. Does this setting make sense? So how does the LSTM obtain the value of the hidden state in the previous state during training?

pascal_notsawo · May 31, 2021, 1:46pm

As can be seen in the source code, the initial hidden state and the initial cell state are all initialized to 0 in this case.

github.com

pytorch/pytorch/blob/master/torch/nn/modules/rnn.py#L662-L671

    
      
          if hx is None:
              num_directions = 2 if self.bidirectional else 1
              real_hidden_size = self.proj_size if self.proj_size > 0 else self.hidden_size
              h_zeros = torch.zeros(self.num_layers * num_directions,
                                    max_batch_size, real_hidden_size,
                                    dtype=input.dtype, device=input.device)
              c_zeros = torch.zeros(self.num_layers * num_directions,
                                    max_batch_size, self.hidden_size,
                                    dtype=input.dtype, device=input.device)
              hx = (h_zeros, c_zeros)