0.4.1 is slower than 0.3.1 in backprop

[rasbt (Sebastian Raschka)]

How much slower? I am wondering because I’ve seen some other post recently where the performance was very bad for PyTorch > 0.4:

Backwards pass runs ~50 times slower on 0.5.0 + GPU versus 0.4.0 + CPU

After much effort, I managed to build and install pytorch on my macbook such that I was able to make use of my GPU, using version 0.5.0a0+ba634c1. I also have an older version 0.4.0 with no GPU support in a different conda environment. I started running my model with the new install and felt it was significantly slower, both using the CPU and the GPU. After timing it, I found that a single backwards pass took 28 seconds using 0.5.0 while the same code took 0.6 seconds with 0.4.0. What could be…