I found the answer. The detach() uses same memory as original tensor. But if I use clone(), it will make independent memory to save value. Therefore, instead of detach(), I should use clone() in this case.
https://discuss.pytorch.org/t/whats-the-difference-between-variable-detach-and-variable-clone/10758