I’ve encountered a problem with GPU memory not being freed while trying to use torchsample predict_loader https://github.com/ncullen93/torchsample/blob/master/torchsample/modules/module_trainer.py#L682
def predict_loader(self, loader, cuda_device=-1, verbose=1): prediction_list =  for batch_idx, batch_data in enumerate(loader): if not isinstance(batch_data, (tuple,list)): batch_data = [batch_data] input_batch = batch_data if not isinstance(input_batch, (list,tuple)): input_batch = [input_batch] input_batch = [Variable(ins) for ins in input_batch] if cuda_device > -1: input_batch = [ins.cuda(cuda_device) for ins in input_batch] prediction_list.append(self.model(*input_batch)) return torch.cat(prediction_list,0)
As far as I understand, current behaviour is correct, because GPU tensors are stored in a list and not freed, so that causes memory to fill up.
I want to save predictions for multiple batches on cpu memory. I’ve tried to modify the code to save the cpu part like this:
but it didn’t help, the GPU memory usage was still rising after every batch.
Current workaround that I use looks like this, but that doesn’t look right :):
pred = torch.from_numpy(self.model(*input_batch).cpu().data.numpy()) prediction_list.append(pred)
Is there a better way to save only a cpu part of a Tensor or sowehow tell pytorch to free GPU memory of a Tensor?