Where is an operation recorded?

Let’s say you execute the following code:
z = x + y
where x and y are Tensors. This should create a computation graph with a single “Add” node. But where in the PyTorch source code is this actually done?

Also, where exactly is this graph stored? We don’t initialize it ourselves. So it’s created behind the scenes?

Provided x or y requires gradients, yes, PyTorch will do that.
You can see it in z.grad_fn and follow it using z.grad_fn.next_functions (and go deeper beyond that).

Best regards

Thomas