Hi,
How do you estimate a minimum GPU requirement for your application ?
This is something I’ve never been able to clarify.
My application will use a model for inference, it’s a pytorch model for OCR (basically I’m using easyocr for this) and I’ll do a single image inference. I’m doing some test on Colab using the following functions after a single prediction:
torch.cuda.max_memory_allocated()
torch.cuda.max_memory_cached()
I always get around 2GB for both (1.9 for the first and 2.5 for the second one), the GPU is a Nvidia T4.
Is this the right way to do it ? Can I safely assume a GPU with 4GB memory is my minimum requirement? Will this still be valid on another GPU model ?