You never need to create zero tensors as the initial hidden state of an nn.LSTM; that’s done automatically. But I’m still confused as to what you’re trying to do with the LSTMs; are you passing in a time series? If so, it needs to be three-dimensional (time by batch by features).