Tensor new dimension in forward method!

autograd
1

I am trying to perform some calculation on the tensor in the forward method of my model, (eg., just adding the even indexed items to the odd ones)
when I print the shape of the tensor directly after entering the forward method everything is normal

print(x.shape)
torch.Size([25, 1, 1, 800])

However, when I add the following print statement (after the the first print)

print(x[2,0,0,0])

I get an error message:

IndexError: index 2 is out of bounds for dimension 0 with size 2

and the output of the first print becomes:

torch.Size([2, 25, 1, 1, 800])

Does anyone have any idea why this happens?
Also, here is the code for my model:

class EXT1(nn.Module):
    def __init__(self, ):
        super(EXT1, self).__init__()
        self.fc1 = nn.Linear(800, 800)
        self.fc2 = nn.Linear(800, 2)
        self.activation = nn.Tanh()

    def forward(self, x):
        print(x.shape,'===============')
        print(x[2,0,0,0])## THIS THE ADDITIONAL PRINT STATEMENT 
        
        x = self.activation(self.fc1(x))
        x = self.activation(self.fc2(x))

        return x
2

Do you see this issue for the last batch?
If you are using a DataLoader, the last batch might be smaller, if the length of your dataset is not divisible by the batch_size without a remainder. In your case, the last batch might only contain two samples, thus you cannot index it using [2, 0, 0, 0].

3

Hi @ptrblck
Thanks for the reply. No it is not the last batch, in fact I realized this issue appears when the model is initialized, but then when I start to load/feed the data to the model, x gets the normal size. Not sure why the additional dimension is added at the initialization.