Pause autograd for a module

Is there a way to make autograd “take a break” for a particular module? Implementing like is done here receives a whole bunch of negative infinities as grad_output.

What do you mean by “take a break”? Does a certain module get Inf gradients and you would like to somehow ignore it?

I’m just not sure why I’m getting Inf gradients. I suspect what might be happening is since I’m using non-differentiable functions inside my custom autograd function, autograd is turning requires_grad off. I don’t entirely understand how custom autograd functions work, though, so that may be a misunderstanding. What I mean by “take a break” is for autograd to not worry about the details of what’s going on inside my custom function; I want it to just trust me and those computations. That is, in the module following it, I want it to just send over values for me to use as grad_output, and for the module preceding it, I want it to just receive the grad_input I send over and assume those are right.
TL;DR, if autograd doesn’t do anything for custom functions then it’s probably a problem on my side; I’m just trying to figure out what’s going on.

Autograd won’t interfere inside your custom forward and backward functions, as it’s disabled by default there.
If you are getting Inf gradients, you might want to debug your code using Anomaly Detection.

I have actually already enabled Anomaly Detection. I receive the following errors:

sys:1: RuntimeWarning: Traceback of forward call that caused the error:
File “demo.py”, line 341, in
outputs = vgg16(inputs)
File “/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 493, in call
result = self.forward(*input, **kwargs)
File “demo.py”, line 264, in forward
x = self.features(x)
File “/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 493, in call
result = self.forward(*input, **kwargs)
File “/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/container.py”, line 92, in forward
input = module(input)
File “/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 493, in call
result = self.forward(*input, **kwargs)
File “/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py”, line 338, in forward
self.padding, self.dilation, self.groups)


Traceback (most recent call last):
File “demo.py”, line 345, in
loss.backward()
File “/opt/anaconda3/lib/python3.7/site-packages/torch/tensor.py”, line 107, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File “/opt/anaconda3/lib/python3.7/site-packages/torch/autograd/init.py”, line 93, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: Function ‘CudnnConvolutionBackward’ returned nan values in its 0th output.

If your first operation is a convolution, I would check the input for NaNs and Infs using torch.isnan and torch.isinf.

I don’t understand. My current structure is of the form:
Lots of ordinary modules from torch.nn->Conv2d->ReLU->ConvMask(my custom module)->ReLU->Conv2d->More ordinary modules from torch.nn
Shouldn’t the backpropagated gradients until the ReLU following the ConvMask already be cleaned of NaNs and Infs, since those are just ordinary modules?