Multi GPU error: RuntimeError: binary_op(): expected both inputs to be on same device, but input a is on cuda:0 and input b is on cuda:7

1

Hi. When I use multiple GPUs and also when I use .cuda() for tensors in the middle of training
I got following error

RuntimeError: binary_op(): expected both inputs to be on same device, 
but input a is on cuda:0 and input b is on cuda:7

which means R2 in on cuda:7 (not sure R2 in on cuda:0 or cuda:7) and R1 is on cuda:0
so, operation is impossible because they’re on difference CPU

R2=torch.where(R2<1e-4,torch.Tensor([1e-4]).squeeze().cuda(),R2)
div_R1_R2=torch.div(R1,R2)

I get same error in the following code
O_img_tc and R_gt_img_tc can’t be calculated because they’re on different GPU

R_gt_img_tc=torch.where(
  torch.abs(R_gt_img_tc)<1e-4,
  torch.Tensor([1e-4]).squeeze().cuda(),R_gt_img_tc)
sha=torch.clamp(torch.div(O_img_tc,R_gt_img_tc),0.0,1.3)[:,0,:,:].unsqueeze(1)

How to solve this, and what am I doing wrong?