I’m confused about overwriting values in pytorch and when you should or should not (link to docs)
I would like some general suggestions if anyone has any, but here are a few examples that I’m confused about:
loss += criterion(output, target) #is overwriting loss acceptable?
for _ in range(5): output = model(input) #Am I overwriting the output? loss += criterion(output, target) loss.backward()
In the example above, I overwrite the Variable
output multiple times in the for loop.
- Am I “overwriting values required to compute gradients” with this in-place operation?
- Does code like this make the
autogradcomputational graph slower in some way?
- What if I was using a Variable with thousands of elements instead of a scalar?