Backward() fails after multiple forward passes when model has spectral_norm

autograd
1

I have 3 different inputs- I preform 3 forward passes and one backward pass.
when my model has spectral_norm in it’s layers, the backward fails with the following message:
one of the variables needed for gradient computation has been modified by an inplace operation
once I remove spectral_norm from my model it passes.

is there any work around for this that will still let me perform 3 forward passes and one backward?

many thanks

2 Likes
2

I think using spectral norm from TORCH.NN.UTILS.SPECTRAL_NORM
instead of TORCH.NN.UTILS.PARAMETRIZATIONS.SPECTRAL_NORM has solved it for me

1 Like
3

I had the same issue with torch.nn.utils.parametrizations.spectral_norm and using torch.nn.utils.spectral_norm instead indeed resolves the problem. Is this a bug?