How to calculate the memory requirement for running inferences

Hi Everyone,
I have trained a deep learning model for image classification.
The trained model size is 250 Mb. How to calculate, how much computation memory, it’ll require to run inferences.
I am trying to run it on my 16GB RAM Macbook Pro, but it is failing after the first iteration.

Hi. I think you can use the PyTorch Profiler here: https://pytorch.org/tutorials/recipes/recipes/profiler.html for getting the amount of memory required per layer, execution time per layer in the model as well as show the amount of memory (used by the model’s tensors) that was allocated (or released) during the execution of the model’s operators.
Hope this helps :slight_smile:

1 Like