Background:
I’m working on an adversarial detector method which requires to access the outputs from each hidden layer (Local Intrinsic Dimensionality Detector - a.k.a.: LID).
I loaded a pretrained VGG16 from torchvision.models
.
To access the output from each hidden layer, I put it into a sequential model:
vgg16 = models.vgg16(pretrained=True)
vgg16_seq = nn.Sequential(*(
list(list(vgg16.children())[0]) +
[nn.AdaptiveAvgPool2d((7, 7)), nn.Flatten()] +
list(list(vgg16.children())[2])))
I looked into the torchvision VGG implementation, it uses the [feature..., AvgPool, flatten, classifier...]
structure.
Since AdaptiveAvgPool2d
layer and Flatten
layer have no parameters, I assume this should work, but I have different outputs.
output1 = vgg16(X_small)
print(output1.size())
output2 = vgg16_seq(X_small)
print(output2.size())
torch.equal(output1, output2)
They are in the same dimension but different outputs.
torch.Size([32, 1000])
torch.Size([32, 1000])
False
I tested the outputs right after the AdaptiveAvgPool2d
layer, the outputs are equal:
output1 = nn.Sequential(*list(vgg16.children())[:2])(X_small)
print(output1.size())
output2 = nn.Sequential(*list(vgg16_seq)[:32])(X_small)
print(output2.size())
torch.equal(output1, output2)
torch.Size([32, 512, 7, 7])
torch.Size([32, 512, 7, 7])
True
Can someone point out what went wrong?
Thank you