Seq2seq Inference slow

siv · November 30, 2017, 10:01am

Hi,
Looks like pytorch is not Optimized for CPU as it is done for GPU. And also on their high priority list to fix the same

You Can refer the below links on the same.

github.com/pytorch/pytorch

Performance of Simple Use Case - PyTorch vs Plain Python with Numpy

opened 05:13PM - 23 May 17 UTC

closed 01:45AM - 26 May 17 UTC

makeyourownneuralnetwork

**Summary - PyTorch seems to be half as fast as plain Python for very simple net…works.** ### Context I am learning PyTorch because it promises convenience and easy access to GPU acceleration. In learning, I re-implemented my previous very simple tutorial 3-layer neural network created from scratch in Python with Numpy. ### Code The two are here: - simple python+numpy: [https://github.com/makeyourownneuralnetwork/makeyourownneuralnetwork/blob/master/part2_neural_network_mnist_data.ipynb](https://github.com/makeyourownneuralnetwork/makeyourownneuralnetwork/blob/master/part2_neural_network_mnist_data.ipynb) - PyTorch: [https://github.com/makeyourownneuralnetwork/pytorch/blob/master/pytorch_neural_network_mnist_data.ipynb](https://github.com/makeyourownneuralnetwork/pytorch/blob/master/pytorch_neural_network_mnist_data.ipynb) ### Results They are built to be as similar as possible - same size, same loss function, same training data, etc The result of 2 timing tests show that the **plain python version is about twice as fast as the PyTorch version**: - home-made simple pure python - 4**40 seconds, 458 seconds** - simple PyTorch version - **841 seconds, 834 seconds** ### More Detail Here's a blog which details the experiment conditions in more detail (near the bottom of the post). - blog [http://makeyourownneuralnetwork.blogspot.co.uk/2017/05/learning-mnist-with-gpu-acceleration.html](http://makeyourownneuralnetwork.blogspot.co.uk/2017/05/learning-mnist-with-gpu-acceleration.html) ### What Am I Doing Wrong? Or does PyTorch have too much overhead for simple scenarios, and only really shines for larger or GPU accelerated cases?

twitter.com/haldaume3/status/900775899431305217

If you are wondering what I did next after this bottleneck. I switched to the tensorflow implementation.