Does the order of with torch.no_grad(): and model.eval() matter?

1

Does the order of with torch.no_grad(): and model.eval() matter?
Let models be saved in a list model_list.

for m in model_list:
    m.eval()
    with torch.no_grad():
        netout = m(x)
        print(netout)

OR

with torch.no_grad():
    for m in model_list:
        m.eval()
        netout = m(x)
        print(netout)

In the latter example we avoid multiple calls of with torch.no_grad().

2

No, the order would not matter here and both codes should yield the same results.
Personally, I would move the no_grad() to the top to indicate that the following code won’t track any gradients.

2 Likes