I have a LSTM network which returns me an output for each input. So when I have an input tensor of shape [seq_length, batch_size, features], it will return me [seq_length, batch_size, hidden_size].
Now I want to apply a fully connected network to each output. However as far as I understand nn.Sequential(nn.Linaer(...)) does only allow to process shapes like [batch_size, features].
However, my input would have the shape [seq_length, batch_size, hidden_size]. And the output shape should than look like [seq_length, batch_size, fc_last_layer_dim]. Is there any way it can process three dimensions in that fashion?
Well, the problem is not with the LSTM, that works just fine. But the fully connected layer applied to the LSTM output can not process the additional dimension. So maybe I’m not understanding correctly what you are suggesting.