Tensor shape for RNN batch training

nlp
#1

Hi,

I want to train an RNN in batches. Suppose:

batch_size = 2
seq_len = 3
input_size = 5

And the data comes in shape of (batch_size, seq_len, input_size). For example:

x = np.array([
    [[0,0,0,0,0], [1,1,1,1,1], [2,2,2,2,2]],
    [[3,3,3,3,3], [4,4,4,4,4], [5,5,5,5,5]]
])

x = torch.from_numpy(x)
# (0 ,.,.) = 
#  0  0  0  0  0
#  1  1  1  1  1
#  2  2  2  2  2
#
# (1 ,.,.) = 
#  3  3  3  3  3
#  4  4  4  4  4
#  5  5  5  5  5
# [torch.LongTensor of size 2x3x5]

Now, I know that I have to reshape it to (seq_len, batch_size, input_size) to be able to feed it to RNN, but I can’t figure how. view() doesn’t do it properly:

x.view(3, 2, 5)
#  (0 ,.,.) = 
#  0  0  0  0  0
#  1  1  1  1  1
#
#  (1 ,.,.) = 
#  2  2  2  2  2
#  3  3  3  3  3
#
#  (2 ,.,.) = 
#  4  4  4  4  4
#  5  5  5  5  5
# [torch.LongTensor of size 3x2x5]

which I think is wrong. I guess the resulting tensor should be like this:

  (0 ,.,.) = 
  0  0  0  0  0
  3  3  3  3  3

  (1 ,.,.) = 
  1  1  1  1  1
  4  4  4  4  4 

  (2 ,.,.) = 
  2  2  2  2  2
  5  5  5  5  5
  1. Am I right?
  2. If yes, how can I do it? Should I initialize a tensor and populate it iteratively or are there better ways?

Thanks

1 Like
#2 
x.permute(1,0,2)
1 Like