Due to compatibility issues, I am using pytorch=0.2.0 with python=2.7
I installed it using conda install pytorch=0.2.0 cuda80 -c soumith as it was pointed out on the forum that this will lead to reduction in lag while using .cuda() for the first time, however I do not see any improvements and loading still takes ~3mins.
Has been some time since PyTorch 0.2, but I can’t remember it being that slow. It should take a few seconds at most, not minutes. Not sure what’s causing the problem in your case, could be that it’s been a bug in PyTorch 0.2?
PyTorch 0.3 and 0.4 also work with Python 2.7, regardign the compatibility issues you mentioned, the Tesla V100 should work with cuda 8 & 9, so I think so if you can install it 0.3 or 0.4 somehow alongside the 0.2 version, it would help figuring out whether it’s a PyTorch 0.2-specific bug or sth else.
I remember this issue occurred if a wrong CUDA version was installed and in the first run it’s recompiling pytorch for your GPU.
Unfortunately there is no CUDA9 for pytorch 0.2.0.
However, could you print torch.version.cuda after your first cuda run?