I’m not sure why you are calculating the loss twice, but note that appending the loss tensor without detaching it will increase the memory usage, since the entire computation graph is attached, and could yield out of memory errors (which users often call a “memory leak”), so you might want to call .detach() on loss2.
3 Likes