I am wondering about the executing time for the forward and backward stages of the convolution 2d operation.
As far as I know, the forward pass is composed by a single convolution operations. When doing the back propagation, the derivatives w.r.t each of the inputs would be computed with a convolution operation each. Thus, I was expecting the backward pass to be slower than the forward.
But, according to the tests I did on the notebook (linked above) the backward pass is just a very small fraction of the forward pass.
From the notebook, I have:
Could someone shed some light on this?