Inference on CPU for detectron2

[Sylvain_Ard]

Hello all !
I have trained my model on GPU with maskrcnn and ResNet 101 on GitHub - facebookresearch/detectron2: Detectron2 is FAIR's next-generation platform for object detection, segmentation and other visual recognition tasks. code and want now to deploy it on CPU for inference. The code is for GPU, how to translate it to CPU please ? I use demo/demo.py for test/inference.
Thank you very much
Best regards

[ptrblck]

I would assume there is no hard-coded dependency on CUDA in the repository so unless you manually push the data and model to the GPU, the CPU should be used.
In case you are using an already provided inference script and cannot see how the GPU is used, mask it via CUDA_VISIBLE_DEVICES="" python inference.py so that PyTorch won’t be able to use it.

[Sylvain_Ard]

OK thank you but for installing the packages for the code, we need to install cudatoolkit ?

[Sylvain_Ard]

and I think there are hard-coded dependencies of CUDA

[ptrblck]

Could you post a minimal, executable code showing this dependency, i.e. the failure on a CPU-only node (or by hiding the GPUs via the aforementioned env variable)?

[Sylvain_Ard]

I tried to install the environment in my device without GPU, but it install cudatoolkit even if I don’t tell it to install and fails with error :
tk-8.6.11 | 3.8 MB | ################################################################### | 100%
lz4-c-1.9.3 | 135 KB | ################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
ERROR conda.core.link:_execute(698): An error occurred while installing package ‘conda-forge::cudatoolkit-11.1.1-heb2d755_7’.
Rolling back transaction: done

LinkError: post-link script failed for package conda-forge::cudatoolkit-11.1.1-heb2d755_7
location of failed script: C:\Users\Sylvain ARD.conda\envs\detectron2\Scripts.cudatoolkit-post-link.bat
==> script messages <==

==> script output <==
stdout:
stderr: Accès refusé.

return code: 1

()

[ptrblck]

I can’t see what exactly is failing, but the install instructions provide CPU-only binaries here, so I don’t think that CUDA is needed.

[Sylvain_Ard]

I fixed this error by doing :

Error access denied

I had finally succeeded by doing : edit : C:\Users\Sylvain ARD\.conda\pkgs\cudatoolkit-10.0.130-h5327add_6\Scripts\.cudatoolkit-post-link.bat : and replace : rem post install EULA message; `pre-link.bat` does not show message and shows warning echo "By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html" >> %PREFIX%\.messages.txt by : rem post install EULA m…

I progress…

[Sylvain_Ard]

I have this error in demo/demo.py :
(detectron2) C:\detectron2-main>python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml --input “datasets/trainval/image (13).jpg” --output output.jpg --confidence-threshold 0.8
[09/21 11:28:20 detectron2]: Arguments: Namespace(confidence_threshold=0.8, config_file=‘configs/COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml’, input=[‘datasets/trainval/image (13).jpg’], opts=[], output=‘output.jpg’, video_input=None, webcam=False)
Traceback (most recent call last):
File “demo/demo.py”, line 117, in
demo = VisualizationDemo(cfg)
File “C:\detectron2-main\demo\predictor.py”, line 35, in init
self.predictor = DefaultPredictor(cfg)
File “c:\detectron2-main\detectron2\engine\defaults.py”, line 281, in init
self.model = build_model(self.cfg)
File “c:\detectron2-main\detectron2\modeling\meta_arch\build.py”, line 23, in build_model
model.to(torch.device(cfg.MODEL.DEVICE))
File “C:\Users\Sylvain ARD.conda\envs\detectron2\lib\site-packages\torch\nn\modules\module.py”, line 673, in to
return self._apply(convert)
File “C:\Users\Sylvain ARD.conda\envs\detectron2\lib\site-packages\torch\nn\modules\module.py”, line 387, in _apply
module._apply(fn)
File “C:\Users\Sylvain ARD.conda\envs\detectron2\lib\site-packages\torch\nn\modules\module.py”, line 387, in _apply
module._apply(fn)
File “C:\Users\Sylvain ARD.conda\envs\detectron2\lib\site-packages\torch\nn\modules\module.py”, line 409, in apply
param_applied = fn(param)
File “C:\Users\Sylvain ARD.conda\envs\detectron2\lib\site-packages\torch\nn\modules\module.py”, line 671, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
File "C:\Users\Sylvain ARD.conda\envs\detectron2\lib\site-packages\torch\cuda_init.py", line 170, in _lazy_init
torch._C._cuda_init()
RuntimeError: CUDA driver initialization failed, you might not have a CUDA gpu.

(detectron2) C:\detectron2-main>pause
Appuyez sur une touche pour continuer…

[Sylvain_Ard]

apparently I need to change cfg.MODEL.DEVICE but what I have to put in ?

[Sylvain_Ard]

I finally succeeded by adding cfg.MODEL.DEVICE = ‘cpu’

[Sylvain_Ard]

this is resolved
thank you

[Aakash_Gupta]

@Sylvain_Ard - whats the impact on your inference time when you run it on a CPU? For my use case the inference time was shooting up (>2-3mins per image vs msec when done on a GPU) Which didn’t make sense for any practical (real-life) deployment

Did you check the D2go library? - GitHub - facebookresearch/d2go: D2Go is a toolkit for efficient deep learning

[Sylvain_Ard]

for me about 40 seconds in CPU and 4 seconds in GPU, but it depends on your CPU model