Hello,
I’m having the same issue since 2 weeks. I’m callling a faster r cnn model, I trained a model with my GPU (only one image by batches because of “memory”). Then I saved the model and now I can call it. But when I only load the model and I’m trying to take a look on the predictions, I use model.eval() and then model([image]) … But it works for 2 images but with 3 images I have a cuda error. But I tried a lot of things : I call torch.cuda.empty_cache() after every prediction, but it didn’t work. Then I add gc.collect() but still doesn’t work. I show you my error :
RuntimeError Traceback (most recent call last)
<ipython-input-4-cb3f5f560ce4> in <module>
----> 1 list_images, list_predictions = inference_gpu([27612, 45125, 31654 ], path_to_video = path_to_video_lab7, draw_bbox = True, threshold = 0.9)
<ipython-input-1-37a3b7b4f6bf> in inference_gpu(list_path_inf, path_to_video, draw_bbox, threshold)
72 model.cuda()
73 model.eval()
---> 74 out = model([image])
75 list_images.append(image)
76 list_predictions.append(out)
~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1050 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051 return forward_call(*input, **kwargs)
1052 # Do not call functions when jit is used
1053 full_backward_hooks, non_full_backward_hooks = [], []
~/anaconda3/lib/python3.8/site-packages/torchvision/models/detection/generalized_rcnn.py in forward(self, images, targets)
94 if isinstance(features, torch.Tensor):
95 features = OrderedDict([('0', features)])
---> 96 proposals, proposal_losses = self.rpn(images, features, targets)
97 detections, detector_losses = self.roi_heads(features, proposals, images.image_sizes, targets)
98 detections = self.transform.postprocess(detections, images.image_sizes, original_image_sizes)
~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1050 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051 return forward_call(*input, **kwargs)
1052 # Do not call functions when jit is used
1053 full_backward_hooks, non_full_backward_hooks = [], []
~/anaconda3/lib/python3.8/site-packages/torchvision/models/detection/rpn.py in forward(self, images, features, targets)
341 # RPN uses all feature maps that are available
342 features = list(features.values())
--> 343 objectness, pred_bbox_deltas = self.head(features)
344 anchors = self.anchor_generator(images, features)
345
~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1049 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1050 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051 return forward_call(*input, **kwargs)
1052 # Do not call functions when jit is used
1053 full_backward_hooks, non_full_backward_hooks = [], []
~/anaconda3/lib/python3.8/site-packages/torchvision/models/detection/rpn.py in forward(self, x)
55 bbox_reg = []
56 for feature in x:
---> 57 t = F.relu(self.conv(feature))
58 logits.append(self.cls_logits(t))
59 bbox_reg.append(self.bbox_pred(t))
~/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py in relu(input, inplace)
1296 result = torch.relu_(input)
1297 else:
-> 1298 result = torch.relu(input)
1299 return result
1300
RuntimeError: CUDA out of memory. Tried to allocate 32.00 MiB (GPU 0; 5.81 GiB total capacity; 4.44 GiB already allocated; 32.38 MiB free; 4.47 GiB reserved in total by PyTorch)
My image’s shape is something like (1060,1020, 3).
Is there a solution to apply inference with a gpu model ?
Thanks !