Cuda out of memory when tensorflow and used with torch

am using Yolov3 by Ultralytics (PyTorch) to detect the behavior of cows in a video. The Yolov3 was trained to detect each individual cow in the video. Each image of the cow is cropped using the X and Y coordinates of the bounding box. Each image then goes through another model to determine whether they are sitting or standing. The second model was also trained with our own dataset. The second model uses Tensorflow and it’s a very simple InceptionV3 model.

However, whenever I try to load both models, I am getting the following errors

RuntimeError: CUDA out of memory. Tried to allocate 50.00 MiB (GPU 0; 16.00 GiB total capacity; 427.42 MiB already allocated; 7.50 MiB free; 448.00 MiB reserved in total by PyTorch) 
If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

If the second model is not loaded then the yolov3 (PyTorch) runs without any issue and it does not even use the whole 16GB of VRAM. Any solution?

7.50 MiB free; 448.00 MiB reserved in total by PyTorch

This indicates that the other application (TensorFlow in this case) is grabbing the entire device memory and doesn’t allow PyTorch to allocate it anymore.
If I’m not mistaken, this is the default behavior in TensorFlow, so you might want to change it via tf.config.experimental.set_memory_growth (I don’t know if this is the proper API and I just found it via a quick search).

1 Like