February 10, 2023, 8:23am
Hi, my custom loss looks like this:
super(CKALoss, self). init()
def forward(self, yhat,local_Y):
ground_truth = local_Y
pred = yhat
m = ground_truth.size()
cka = CKA(m=m, kernel='linear')
#rho = cka.compare(pred.detach().cpu().numpy(), ground_truth.detach().cpu().numpy())
rho = cka.compare(pred, ground_truth) #centered kernel alignment similarity
rho = torch.from_numpy(np.asarray(rho))
loss = torch.log(1-rho)
Then I got the error message “TypeError: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.”
Can anybody know how to fix it?
February 10, 2023, 8:47am
TypeError: can’t convert cuda:0 device type tensor to numpy.
The error says that you initialized your tensor in CUDA memory and need to convert the tensor to CPU tensor before changing to numpy arrays. FYI: NumPy does not have CUDA support.
You can initialize your tensor in the CPU memory from the beginning. Something like this:
torch.tensor([1, 2, 3], device="cpu")
This would resolve your error and make your code much cleaner
A small question: Why are you converting tensors to NumPy arrays? I think there’s no need; PyTorch has full support.
There is a tensor that its in cuda and not in cpu. Numpy does not support cuda operations so you need to transform it back to cpu. Probably it is the rho value, just stick a .cpu() inside of the call np.assarray
try below if you want to use your gpu
add this code
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
and attach .to(device) to your model and local_Y tensor or numpy array like this
model = Model(~~).to(device)
local_Y = ~~.to(device)
If it’s not work, CKA could make this error. find cuda setting of CKA class…