torch.Tensor.scatter_add_ error in backward function

autograd
1

I have a tensor of zeros who’s values I’m filling using the scatter_add_ function like so -

y.scatter_add_(dim=2, idxs, vals)

where

y -> (batch_size, channels, N’)
idxs -> (batch_size, 1, N)
vals -> (batch_size, 1, N)
where N != N’.

This op is followed by a reshape of y into shape (batch_size, channels, H x W) where N’ = H x W.

I get the following error when I try to compute the gradients -
Traceback (most recent call last):
File “features_train.py”, line 152, in
main()
File “features_train.py”, line 144, in main
engine.train()
File “/home/siddhesh/LuminFlowModels/LidarML/pytorch_src/training_engine.py”, line 66, in train
loss.backward()
File “/home/siddhesh/pytorch_nightly/lib/python3.5/site-packages/torch/tensor.py”, line 102, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File “/home/siddhesh/pytorch_nightly/lib/python3.5/site-packages/torch/autograd/init.py”, line 90, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: invalid argument 2: Input tensor must have same size as output tensor apart from the specified dimension at /pytorch/aten/src/THC/generic/THCTensorScatterGather.cu:27

Could someone explain why I’m getting this error? Or more specifically what tensors are referred to in the Runtime Error when it says ‘input tensor’ and ‘output tensor’ that are supposed to have the same size.

2

Hi,

I am surprised that it works in the forward, given that you index on dim2.
dim0 and dim1 should be the same for scatter but your dim1 is 1 in one case and channels in the other.
Could you try expanding idxs and vals to (batch_size, channels, N) before giving them to the scatter_add_? Does that fix the problem?

3

My bad. The shape of vals is (batch_size, channels, N). The problem was fixed by expanding idxs to (batch_size, channels, N).

Thank you!