I have figured out the real problem. This was my work PC which have security guidelines regarding installation of PIP packages in Python. My company had their own repository for PyPi packages and I index this link during pip installations. When I went into the repository, I found that all the torch installations available in the company repo are CPU binaries. So, even if i use the command to install CUDA binary, only CPU binary was being installed.
So I have installed the pip packages from my home network without connecting to company VPN. 
This directly installed torch compiled with CUDA.
Thank you very much for all the help. 
May Allah bless and guide you. 