Slow pytorch inference speed

Recently I was experimenting with different object detection techniques faster than mobilnet SSD v1 tensorflow api. I came across similar implementation in pytorch. But while execution I found that time taken for pytorch implementation(0.1 sec) is more than to that with tensorflow OD(0.06 sec). I think all frameworks should give similar results for same algorithm.

Hi,

You will need to give more background on the hardware you’re using to get a better idea.
Also this is a third party implementation so it might not be very well optimized. compared to the builtin version provided by tensorflow.

1 Like

Thanks for your reply, How can I optimize PyTorch implementation? Can you please share some references I can look at to get the more satisfying results? Also, is it true that a fully optimized code in PyTorch will give the same results to that with Tensorflow’s OD?

PS: Also I think the main purpose of these frameworks is that we should more focus on implementation rather than optimization

How can I optimize PyTorch implementation?

I am not familiar with this particular implementation.
You can look online for “pytorch in production” and you should get all the materials you need.

Also, is it true that a fully optimized code in PyTorch will give the same results to that with Tensorflow’s OD?

This is not guaranteed. They will most likely be very similar because all the heavy lifting is done by cudnn and BLAS libraries anyway. But there might be some differences for less traditional architectures.

Also I think the main purpose of these frameworks is that we should more focus on implementation rather than optimization

I think this is a difference between pytorch and Tensorflow yes. While you can find many many things internalized and optimized in Tensorflow, it’s not the case for pytorch. We only add the very widely used methods to the core. You can find a bit more in the specialized libraries like torchvision or torchtext.