Is there any way to implement stochastic pooling?

1

I wonder if it is possible to implement stochastic pooling as in this paper, “Stochastic Pooling for Regularization of Deep Convolutional Neural Networks” or “fractional average pool” as in Tensorflow?
If possible, what is the most efficient way? Do I need to add custom C module?
(It is nice if it’s possible with only python modules and not too slow…)

2

Hello,

thank you for asking!

Stochastic pooling as in the paper with stride = pool size is easy to implement using view (so that the indices to be pooled are in their own dimension e.g. x.view(x.size(0),x.size(1)//2,2)), sampling random coordinates from multinomial and using that for indexing.
The weighting can be done using a standard (“spatial”) convolution in the functional interface and a filter that contains the probability.

You could also use stochastic average pooling by drawing scores + softmax + convolution similar to what they suggest for test time but with random weights.

I could do an implementation example if that helps.

Someone more knowledgable than me will have to answer fractional average pooling (fraction max pooling is done in pytorch, but I’m not aware of average pooling).
An ad-hoc way of achieving something similar could involve “splitting the activations at the border” of larger kernels by using positive convolutions stencils that sum to one but are not all equal and then keeping a subgrid.

Best regards

Thomas

3

Thanks for the answers!

As for your first suggestion, is it possibly extend to 2D with shape (N,C,H,W)?
If its possible, should x be reshape to x.view(N,C,H//2,W//2,2,2) or x.view(N,C,H//2,2,W//2,2)?
(I’m not sure but the correct one might be x.view(N,C,H//2,2,W//2,2).permute(0,1,2,4,3,5)?)

4

You need to keep the 2s next to the original dims, so the latter variant. Instead of permuting, I’d probably either sample the two independently (if the desired distribution is just the product) or do the //2 and %2 calclations to get the indices from values 0…3. (Numpy has unravel, but that is CPU only and a bit overkill here.)

Best regards

Thomas

5

Can you please explain the process by giving an example?

1 Like
6

Is there any code available for this? I am working on stochastic pooling. However, still not much idea about how to implement it.

1 Like
7

Did u implement stochastic pooling?If you get can u plz provide with an example