Will the same model input data twice retain the gradient information of the first input data?

input1 = #sth
input2 = # sth
label1 = # sth
label2 = # sth
output1 = model(input1)
output2 = model(input2)
loss1 = criterion(output1, label1)
loss2 = criterion(output2, label2)
total = loss1+loss2
total_loss.backward()
optimizer.step()

I wonder will the backward() function consider the effect of input1?

Yes, it should (if total.backward()). Try to print and see if they are different?
Since the backward is on total i.e loss1+loss2, the computation graph would include both 1, 2 inputs.

You could also refer the GAN tutorial where something similar is done

1 Like