What is the difference between adaptive_avg_pool2d and avg_pool2d under torch.nn.functional? What does adaptive mean?
In avg_pool2d, we define a kernel and stride size for the pooling operation, and the function just performs that operation on all valid inputs. For example, an avg_pool2d with kernel=3, stride=2 and padding=0, would reduce a 5x5 tensor to a 3x3 tensor, and a 7x7 tensor to a 4x4 tensor.(HxW)
In adaptive_avg_pool2d, we define the output size we require at the end of the pooling operation, and pytorch infers what pooling parameters to use to do that. For example, an adaptive_avg_pool2d with output size=(3,3) would reduce both a 5x5 and 7x7 tensor to a 3x3 tensor.
This is especially useful if there is some variation in your input size and you are making use of fully connected layers at the top of your CNN.
Thanks Mazhar! Based on what you said, it seems to me ‘adaptive’ is in the sense of adapting the kernel size and stride and maybe padding to the output size, not in the sense of varying the weights while taking the average. In other words, the average is the plain average (sum divided by the number of elements in the kernel), not a weighted average of elements that fall within the kernel. Is my understanding correct?
That’s correct, LMA, to the best of my knowledge.