A strange problem with torch.optim: the variable changes, but the difference between the old and the updated variable is shown to be zero

autograd
1

I stumbled upon a rather weird issue while debugging a larger code in PyTorch. A simple instance to reproduce the issue is here: https://colab.research.google.com/drive/11kBaxMOxN9i0X1vtaX47Yz7rkkHz7Kpu

As you can see, the variable x gets updated in every epoch (see that their norms are different), however, the norm of their difference (rel_diff) turns out to be zero. Any help is deeply appreciated.

2

Use x_old = x.detach().clone(), since .data will be a reference and point to the same location in x_old and x_new.