How to know the exact GPU memory requirement for a certain model?

[feiyuhuahuo]

I just comment and uncomment cudnn.benchmark=True to test the memory usage. And I found they differ far. I don’t know the exact mechanism behind it.
Here is an answer from someone else.

What does cudnn.fastest = True work?

It just signals the Pytorch to use the fastest implementation available for operations such as Convolution etc. when enabled, they usually consume more memory (that is cudnn.benchmark and cudnn.fastest)