a = torch.zeros((1,2,100))
a.expand(1,4,100) gives error
RuntimeError: The expanded size of the tensor (4) must match the existing size (2) at non-singleton dimension 1
Why is this?
Also i am trying to do batched beam search, my batch size=2 and beam size=2. When it comes out from encoder, hidden dimension is 1x2x100 [as i dont consider beam there]. Now as it has to be fed into the decoder with two initial states for two sentences, i need to make it 1x4x100. Is the approach right?
From the above example its clear that it returns new view of the exists.
In the first case, it didn’t throw any error because it’s just of shape [1,1,100] from which you can produce anything as its single shape almost with no dimensions.
Not the same in the second case.
Decoder and Encoder(My Thinking)
Hidden dimension is 1x2x100 : 2 indicates the batch size I think and you don’t need to pass anything 14100 as the pytorch net itself will handle this.
For the Decoder and Encoder , i also thought this initially
But going forward we would have 2 hidden states for each sentence [as there are two sources per sentence[beam size=2] each giving a new hidden state . Which dimension would pytorch put the 4 hidden states into?
I would need to feed four hidden states [2 sentences;2 beam size] which axis’s dimension should be increased?