Torch.no_grad() doesn't result in speed boost

autograd
1

I thought with gradient calculations involved, the inference time must be fairly faster. Am I misunderstanding something here?

import torch
from torchvision.models import vgg16

vgg = vgg16(pretrained=True)
x = torch.randn(1, 3, 480, 640)

%timeit vgg(x)
# 1.36 s ± 93.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%%timeit
with torch.no_grad():
    vgg(x)
# 1.88 s ± 294 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
2

I think in your test, the forward loop does not involve gradient calculations.
Thus the speed does not change much if no backward() is called, it may just save you some memory of the gradient placeholders.

1 Like
3

that sounds reasonable, thanks