Here are two cases use “=” and “clone”:
x1 = torch.randn(4)
y1 = x1
z1 = y1.mean().backward()
x2 = torch.randn(4)
y2 = x2.clone()
Since x1 and x2 can get their grads (which are equal), and my questions are:
- what’s the difference in the two cases?
- which one is preferred when creating a computation graph?
= is simply setting y1 to point to the object that x1 points to, so y1 and x1 both points to the same Python object.
Clone made a copy to x2, so even though they are still equal, they aren’t the same object. y1 points to the cloned object while x1 points to the original one.
Thanks for the explanation! And which way is better when create a computation graph in your option?
It depends on your purpose because they have different behavior. If you are only doing simple calculations, I suggest (1). One thing to note is that because (2) uses
clone, (2) uses more memory.