Then to use .clone() in optimizer computations?

Should anyone explain why here clone() used?

Whats general rule about it?

As far as I understand, this command copies the variable, in case I later modify it.
But in this case, this variable is not used further.
Is this an error in the code?

The clone() might not be necessary.
Did you try to remove it and compare the output/training to the original implementation?