Custom loss: TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first

Hi, my custom loss looks like this:

class CKALoss(torch.nn.Module):
“”"
“”"
def init(self):
super(CKALoss, self).init()

def forward(self, yhat,local_Y):
    ground_truth = local_Y
    pred = yhat
    m = ground_truth.size()[0]
    cka = CKA(m=m, kernel='linear')
    #rho = cka.compare(pred.detach().cpu().numpy(), ground_truth.detach().cpu().numpy())
    rho = cka.compare(pred, ground_truth) #centered kernel alignment similarity
    rho = torch.from_numpy(np.asarray(rho))
    loss = torch.log(1-rho)
    return loss

Then I got the error message “TypeError: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.”
Can anybody know how to fix it?

Hi, there!

TypeError: can’t convert cuda:0 device type tensor to numpy.

The error says that you initialized your tensor in CUDA memory and need to convert the tensor to CPU tensor before changing to numpy arrays. FYI: NumPy does not have CUDA support.

You can initialize your tensor in the CPU memory from the beginning. Something like this:

torch.tensor([1, 2, 3], device="cpu")

This would resolve your error and make your code much cleaner :slight_smile:

A small question: Why are you converting tensors to NumPy arrays? I think there’s no need; PyTorch has full support.

Thank you!

There is a tensor that its in cuda and not in cpu. Numpy does not support cuda operations so you need to transform it back to cpu. Probably it is the rho value, just stick a .cpu() inside of the call np.assarray

try below if you want to use your gpu

add this code
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

and attach .to(device) to your model and local_Y tensor or numpy array like this

model = Model(~~).to(device)
local_Y = ~~.to(device)

If it’s not work, CKA could make this error. find cuda setting of CKA class…