I just updated my pytorch to 1.3 and it takes a long time (5~10mins) to call cuda() on my quite large model. Before the update, its almost instantaneous. I am using a titan x pascal with cudatoolkit 10.1.168 if that helps.
Once the model is loaded, the training itself seems fine.
I am also experiencing this issue. I posted an issue to the pytorch github. The quick fix is to downgrade to CUDA 10.0
conda install -c pytorch pytorch cudatoolkit=10.0
not entirely sure why yet, but we’re looking into it.
tracking in: https://github.com/pytorch/pytorch/issues/27807