Is RNN model time major or batch major

I think RNN model in Pytorch is batch major, the shape of input data should be ( n_batch, n_step, input_size). But in the demo code about the word_language_model https://github.com/pytorch/examples/blob/master/word_language_model/main.py , it seems that the data input to rnn model is time major, which shape is ( n_step, n_batch, input_size). This confused me. Could anyone explain it? Thanks

batch_first flag controls that.

1 Like

The LSTM model does have batch_first flag, but the RNN model implemented in torch.nn does not have batch_first flag.

What do you mean? All recurrent layers have. Here is the one for nn.RNN http://pytorch.org/docs/master/nn.html#torch.nn.RNN

1 Like

Thanks. I found it finally after upgrading my torch package. :slight_smile: