Column-wise operation and gradient

Hi,
I’m trying to manipulate a matrix by multiplying each column by a matrix.
My code looks like this:

for i in range(s.shape[0]):
	s[i,:] = torch.matmul(s[i,:],M)
return s

M and s are m*m Tensors.

When I try to backward, I get this error:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1, 81]], which is output 0 of UnsqueezeBackward0, is at version 81; expected version 80 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

and

…\torch\csrc\autograd\python_anomaly_mode.cpp:57: UserWarning: Traceback of forward call that caused the error:

point to this block of code.

What am i doing wrong, and how can I modify this ?

Two possible ways:

  1. Use batch matrix multiplication (or einsum or whatever).
  2. Instead of overwriting s[i], collect the results in a list and do torch.stack on it after the for loop.

Best regards

Thomas

1 Like

What would the einsum look like ? I’m struggling a bit understanding the syntax

What are your input shapes?

Best regards

Thomas