Data parallel with rnn

[agadetsky (Artyom)]

Multi layer RNN with DataParallel

@Varg_Nord I’ve tested approach with hidden state permutations. For me it’s faster than using batch_first=False and using necessary permutations in .forward. Now, correct RNNLM with DataParallel looks following: class Model(nn.Module): def __init__(self, ntokens=100000, nx=300, nhid=600, nlayers=3, dropout=0.5): super(Model, self).__init__() self.keep_prob = dropout self.ntokens = ntokens self.nx = nx self.nhid = nhid self.nl…