After following this post How to debug causes of GPU memory leaks? - #9 by Carson_McNeil, I was able to identify the cause of the memory leak. It seems that the input to my model was being accumulated, resulting in an out-of-memory issue. Here is my log output after printing out allocated tensor.
Still, I have not been able to resolve it yet. Any suggestions would be appreciated.