Torch.no_grad()

autograd
1

In the past, we were able to separate a variable from the current state of a network by doing:

states = Variable(torch.Tensor(width, height), volatile=True)

It seems there is an API change to with torch.no_grad() but I could not find the documentation on docs pages.

It would seem that this is the correct way not to calculate gradients on a variable going forward:

        >>> x = Variable(torch.Tensor([34, 54]), requires_grad=True)
        >>> with torch.no_grad():
                    y = x * 2

Am I correct?

7 Likes
2

Yes, the reason that it is not on the doc yet is that it is not in any official release. We will make sure to document such things properly before a new release.

3

It got pulled when I compiled a docker image this evening.

People can already see it. The earlier the documentation is updated, the better.

1 Like
4

We should definitely update it soon. Thanks for reminding us. I will ask people who are more familiar with the change to write the doc.

That said, you are compiling from github master, i.e., not a release. Although we are usually pretty good at keeping the doc in sync with master, there can sometimes be some delays.

2 Likes
5

I faced same issue, not only it gives warning it also consumes much more memory during validation-- because it computes grads unnecessarily. However, with little code change as shown in the post, it fixes both issues.

1 Like
6

for what version is this out? It seems that 0.3.1 yields errors:

AttributeError: module 'torch' has no attribute 'no_grad'
7

it’s only on master.

8

If I have code written by someone else (see source), is there a way to replace this with something that is not in master?
I can’t use master because it won’t run with my CUDA compute 5.0 GPU.

9

I’m having the same issue @Brando_Miranda

10

The function was added in 0.4.0. You are probably using an older version of PyTorch.
You can find the install instructions on the website.

1 Like
11

I am having the same warning of using:
data, target = Variable(data, volatile=True), Variable(target)

should it be changed to:

with torch.no_grad():            
     data, target = Variable(data), Variable(target)
4 Likes
12
  1. You shouldn’t use Variable wrappers anymore.
  2. Use torch.no_grad() to wrap around the entire inference code. (or use torch.set_grad_enabled）
2 Likes
13

Can u please show how to use with torch.no_grad():

And replace volatile

I am testing a GitHub project and they have used volatile =False as a function argument
So do I need to replace it with requires_grad=True

14

In fact,you can read the official document,the torch.no_grad() in the part of doc which called torch.autograd,https://pytorch.org/docs/stable/autograd.html#locally-disable-grad
In addition,the Variable() function return a torch tensor,which is equal tensor(requires_grad=False)

The Variable API has been deprecated: Variables are no longer necessary to use autograd with tensors. Autograd automatically supports Tensors with requires_grad set to True . Below please find a quick guide on what has changed:

  • Variable(tensor) and Variable(tensor, requires_grad) still work as expected, but they return Tensors instead of Variables.
    image
    image750×1334 145 KB
2 Likes