Pickle error when saving pandas dataframe

1

Hi,

I get this error:

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device(‘cpu’) to map your storages to the CPU.

I’m not sure if I am saving my outputs incorrectly, however, I changed the last line in the below code from df.to_csv to df.to_pickle, in order to save the predictions in a different format. When saving into a csv file I do not get errors, however would prefer to pickle/ dump into an npy file. Any suggestions on what I should do to resolve this error?

predictions = []
imgpaths = []
probabilities = []
true_labels = [] 

with torch.no_grad():
     for i,(inputs10x, paths10x, labels) in enumerate(dataloaders_dict['val']):

            test_inputs10x = inputs10x.to(device)
            test_labels = labels.to(device)
            test_paths = paths10x

            test_x10 = densenet(test_inputs10x)

            outputs = classifier(test_x10)
            test_loss += criterion(outputs, test_labels).item()
            _, pred = torch.max(outputs, 1) #pred = torch.max(outputs, 1)[1]

            prob = F.softmax(outputs, dim=1)
            top_p, top_class = prob.topk(1, dim = 1)

            for q in top_p:
                probabilities.append(q)

            for x in pred:
                predictions.append(x)
            
            for z in test_paths:
                imgpaths.append(z)
            
            for l in test_labels:
                true_labels.append(l)


b = {'paths': imgpaths, 'predicted_labels': predictions, 'true_labels': true_labels, 'probabilities': probabilities}
df = pd.DataFrame.from_dict(b)
df.to_pickle(args.npy_file  + '.pickle')
2

I don’t quite understand why this error is raised in the posted code, as it’s pointing to a deserialization in a non-CUDA installation.
Based on the code snippet I assume you might be using CUDA tensors? If so, could you push them to the CPU before trying to store them via pandas?

1 Like