How to implement stochastic pooling in pytorch?

smani · January 11, 2020, 4:17pm

I am trying to implement Stochastic pooling. But I am very much unclear about how to sample from a multi-nominal distribution to get the location from where we need to pool. Could you please provide any example code?

Eta_C · January 13, 2020, 3:10am

Are you looking for this?

github.com

renebidart/adv-denoising/blob/1400e147ee517667fb9aeb67fa3b38a233c4ab3b/models/StochasticPool2d.py#L61-L78


# def gen_random(values):
#     if torch.sum(values) != 0:
#         idx = torch.multinomial(values, num_samples=1)
#     else:
#         idx = torch.randint(0, values.shape[0], size=(1,)).type(torch.LongTensor)
#     return values[idx]


# init_size = x.shape


# x = F.pad(x, self._padding(x), mode='reflect')
# x = x.unfold(2, self.kernel_size[0], self.stride[0]).unfold(3, self.kernel_size[1], self.stride[1])
# x = x.contiguous().view(-1, 4)


# x = torch.stack([
#          gen_random(x_i) for i, x_i in enumerate(torch.unbind(x, dim=0), 0)
#         ], dim=0)
# x = x.contiguous().view(init_size[0], init_size[1], int(init_size[2]/2), int(init_size[3]/2))
# return x

smani · January 15, 2020, 4:21pm

Thank you very much for the implementation. Do you think the run time of this can be somehow improved, for e.g. by somehow replacing the for loop?

Nirthika_Rajendran · January 17, 2020, 4:11am

Hi, In the stochastic pooling, the author used probabilistic weighting at test time. I tried that, but the testing is not performing well. Did you have any solution for that?

Eta_C · January 17, 2020, 5:40am

Hi, @smani @Nirthika_Rajendran
Sorry I haven’t studied this code carefully. I just happened to see it before, so I shared it.