I am writing a custom module in which the Function will process the input and create buffers in Function scope to hold the processed data for backpropagation.
During the evaluation/test/inference phase of the model, I see that the memory usage keeps on increasing for the model. I am not sure about the reason.
I have the below questions:
- Whether the reason could be: Since the backpropagation is not done during inference, the buffers in the Function are not deallocated?
- How long the ‘Function’ created in the custom ‘Module’ be in memory? Is it until the backpropagation?
- I want to remove the buffer data in ‘Function’ during inference cases. Is there any way to achieve this?