Conv2D: Forward way slower than Backward

Dear fellows,

I am wondering about the executing time for the forward and backward stages of the convolution 2d operation.

Check this notebook

As far as I know, the forward pass is composed by a single convolution operations. When doing the back propagation, the derivatives w.r.t each of the inputs would be computed with a convolution operation each. Thus, I was expecting the backward pass to be slower than the forward.

But, according to the tests I did on the notebook (linked above) the backward pass is just a very small fraction of the forward pass.

From the notebook, I have:

Could someone shed some light on this?


I think you’re missing the following line :wink:

    backward1 *= 1e3 # From seconds to miliseconds

Then all the timings will be what you expect.

Thank you so much. I can’t believe I missed that. It’s very embarrassing, what a shame.

No worries! Took me a while to find it as well :wink: