Should the requires_grad=False of model weights when using torch.inference_mode


As far as I understanding, when using the torch.inference_mode() or torch.no_grad(),
the model will not caculating the gradients and the requires_grad will be False.
I write a simple code to reproduce my question

import torch
import torch.nn as nn

class Network(nn.Module):
    def __init__(
        self.conv = nn.Conv2d(2, 32, (3, 3))

    def forward(self, x):

        with torch.inference_mode():
            x = self.conv(x)

        return x

x = torch.randn(4, 2, 257, 513)
backbone = Network()

The corresponding outputs:


The output requires_grad of x is False, but the requires_grad of self.conv.weight is True.
Is the behavior of the latter correct or there might be something that I haven’t noticed?
Thank you for your reply in advance.

This is expected since the inference_mode context won’t change the .requires_grad attributes of parameters or inputs. The output won’t be attached to a computation graph and no intermediate tensors will be stored, as expected.

1 Like

Ok, I got it. Thanks for your reply!

Quick clarifying question from a newbie. Does it mean that inference_mode would ignore all .requires_grad attributes on parameters or inputs? Would setting .requires_grad of parameters and inputs to False improve performance within inference_mode?

No, it won’t improve the performance as the gradient calculation is already disabled in inference_mode.

1 Like