Hi everyone,
I am using minibatch to accelerate my rnn network. The size of my minibatch is 50.
Now I get the following code:
hiddens, hidden = self.rnn(vids_embeddings_sorted_packed, hidden)
hidden goes well. It owns all 50 results.
result:
hidden
tensor([[[ 0.7852, -0.7907, 0.0129, …, -0.5274, -0.6168, 0.8388],
[-0.1980, 0.5532, -0.4744, …, 0.3106, -0.3343, 0.1624],
[ 0.2504, 0.1677, -0.5666, …, -0.6386, 0.0573, 0.2041],
…,
[ 0.9051, -0.7725, 0.5131, …, 0.8861, 0.7480, -0.1111],
[ 0.7791, -0.8842, -0.6890, …, -0.2500, -0.7887, 0.4737],
[-0.1205, 0.2851, -0.5888, …, 0.2452, -0.1945, -0.8261]]])
hiddens should own all the intermediate results. But it only owns the intermediate results of the first one, but without the remaining 49 ones.
result:
hiddens
PackedSequence(data=tensor([[ 0.8669, -0.2405, -0.2589, …, -0.2067, 0.3336, 0.6909],
[-0.3318, 0.8954, -0.5930, …, -0.3042, 0.3690, -0.4933],
[ 0.7121, 0.0110, 0.2848, …, 0.5082, 0.3286, -0.2838],
…,
[ 0.1060, -0.9560, 0.0281, …, 0.7583, 0.7072, 0.8017],
[ 0.8482, 0.2802, -0.2152, …, 0.8695, 0.1836, -0.9567],
[ 0.7852, -0.7907, 0.0129, …, -0.5274, -0.6168, 0.8388]]), batch_sizes=tensor([ 293, 293, 111, 46, 21, 12, 7, 4, 3, 3,
3, 2, 2, 1, 1, 1, 1, 1, 1, 1,
1]))
You may find that the last row of hiddens should end in -0.8261, but it only contains the result of the first row of hidden, but neglect the result of the remaining 49 row of hidden.