Slow pytorch on V100

2

Did you make sure that the GPU is being utilized?
Check nvidia-smi to see the usage

$ nvidia-smi