Are the hidden_state,cell_state reset for each element/time-step of LSTM?

nlp
1

I have this configuration of my encoder :

  1. class Encoder_LSTM(nn.Module):

  2. def __init__(self,dict_config = dict_config):

  3.     super(Encoder_LSTM, self).__init__()

  4.     self.input_size = dict_config["input_dim"]

  5.     self.hidden_size = dict_config["hid_dim"]

  6.     self.output_size = dict_config["output_dim"]

  7.     self.num_layer = dict_config["num_layer"]

  8.     self.Encoder_LSTM = nn.LSTM(self.input_size, self.hidden_size , batch_first=True , num_layers=self.num_layer , dropout=0.2 )

  9. def forward(self, input):

  10.     self.batch_size = input.shape[0]

  11.     self.hidden_init = torch.zeros(self.num_layer, self.batch_size, self.hidden_size, device=device)

  12.     self.h_0 = self.hidden_init

  13.     self.c_0 = self.hidden_init

  14.     self.output,  (h,c) = self.Encoder_LSTM(input, (self.h_0, self.c_0))

  15.     return self.output

I think I am a bit confused about the “forward” method. It seems like every time you call it, it initialize the hid_state/cell_state. Now, I was wandering , when you have a sequence , is the forward method call on each timestep? If yes, wouldn’t it mean that for each time step the hid_state/cell_state are reset ?

1 Like