@Ke_Bai You might want to read this post https://discuss.pytorch.org/t/problems-with-weight-array-of-floattensor-type-in-loss-function/381. I too had the issue where both the network and inputs were on GPU, causing the error. But the input type was DoubleTensor, however it should have been FloatTensor instead.