What does it mean to move a loss function to device (GPU)?

If the model predictions and targets are already on the GPU, I would keep them there, since the cpu() operation would synchronize the code, and might lower the performance.

2 Likes