How can one modify a model during training without affecting the computation tree that does backprop?

[Brando_Miranda (MirandaAgent)]

Actually, do you mind spelling out one detail that is particularly confusing to me? When we loop and do SGD, when exactly are new computation graphs created? I think thats the part that is confusing to me and even made me post the following question:

Why is the gradient update not accidentally included as part of the computation graph?

I was trying to understand how pytorch decides what operation and which not to include in the computation graph that eventually does backprop. For the simplest example I could think, consider a (maybe ugly) manual implementation of SGD: for i in range(nb_iter): # Forward pass: compute predicted Y using operations on Variables batch_xs, batch_ys = get_batch(X_train,Y_train,M,dtype) # [M, D], [M, 1] ## FORWARD PASS y_pred = mdl.forward(batch_xs) ## LOSS batch_loss = (1/M)…

however it seems that the answer to is is just “acting on mdl.W.data does not add things to the computation graph”. Then when is the computational graph formed?