Is this code correct or what arguments should I pass to the adaptiveavgpool2D and softmax? I suspect it is wrong since it is giving me lower than expected accuracy
self.layer_attend1 = nn.Sequential(nn.Conv2d(64, 64, stride=2, padding=1, kernel_size=3),
nn.AdaptiveAvgPool2d(1),
nn.Softmax(1))
Mainly I think the args to nn.Softmax is wrong here. Given the code is running and throws no error I find it hard to debug.
The docs describe each input argument (nn.AdaptiveAvgPool2d, nn.Softmax) so you can see that the former is using the argument as the output_size while the latter uses it as the dim argument.
In case you are unsure what these arguments do, write a small code snippet to check its usage, e.g. via:
pool = nn.AdaptiveAvgPool2d(1)
x = torch.randn(2, 3, 4, 4)
out = pool(x)
print(out.shape)
> torch.Size([2, 3, 1, 1]) # returns an output with the specified output_size
sm = nn.Softmax(1)
out = sm(x) # applies to softmax in the specified dimension
print(out.sum(1))
> tensor([[[1.0000, 1.0000, 1.0000, 1.0000],
[1.0000, 1.0000, 1.0000, 1.0000],
[1.0000, 1.0000, 1.0000, 1.0000],
[1.0000, 1.0000, 1.0000, 1.0000]],
[[1.0000, 1.0000, 1.0000, 1.0000],
[1.0000, 1.0000, 1.0000, 1.0000],
[1.0000, 1.0000, 1.0000, 1.0000],
[1.0000, 1.0000, 1.0000, 1.0000]]])
