Is the computation graph created every time you evaluate your model on an input?
Yes, the computation graph will be created while running the specified operations (usually in your model’s forward, but it can be created by any arbitrary operation with a tensor which requires gradients).
If you don’t specify retain_graph=True, the computation graph will be cleared after the backward() call.