Extract the 2048 vector of a fine-tuned Inception V3 on test set

[Mona_Jalal]

I have already finetuned an Inception V3 model on my dataset which was already pretrained on ImageNet.

Now, I need to load the save pt model and run it on test patches and extract the 2048 feature vector from Inception V3.

I am not sure what code snippet I should use for extracting feature vectors of size 2048 on an inception v3 model that is already finetuned on my own data (train data) on the test set.

import torch
model = torch.load('pt/model_ft_100e_best_acc.pt')
feature_extractor = torch.nn.Sequential(*(list(model.modules())[:-2]))
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

batch_size = 1

input_size = 299 # for Inception V3

data_transforms = {
    'train': transforms.Compose([
        transforms.Resize((input_size, input_size)),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.6964, 0.4764, 0.6246], [0.1896, 0.2210, 0.1864]) 

    ]),
    'val': transforms.Compose([
        transforms.Resize((input_size, input_size)),
        transforms.ToTensor(),
        transforms.Normalize([0.6964, 0.4764, 0.6246], [0.1896, 0.2210, 0.1864]) 
    ]),

    'test': transforms.Compose([
        transforms.Resize((input_size, input_size)),
        transforms.ToTensor(),
        transforms.Normalize([0.6964, 0.4764, 0.6246], [0.1896, 0.2210, 0.1864]) 
    ])
}


data_dir = "/projectnb/ivcgroup/jalal/data/fold1/" # for running on SCC
# Create training and validation datasets
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x]) for x in ['test']}
# Create training and validation dataloaders

dataloaders_dict = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=batch_size, shuffle=True, num_workers=4) for x in ['test']}


sample_fnames_labels = dataloaders_dict['test'].dataset.samples
sample_large_images = {}


test_large_images = {}
test_loss = 0.0
test_acc = 0


with torch.no_grad():
    test_running_loss = 0.0
    test_running_corrects = 0
    print(len(dataloaders_dict['test']))
    for i, (inputs, labels) in enumerate(dataloaders_dict['test']):
        print(i)
        test_input = inputs.to(device)
        test_label = labels.to(device)
        test_output = feature_extractor(test_input) # here I need to get feature vectors of size 2048
        print("test feature size is: ", test_output.shape)

[Mona_Jalal]

In my model, I do see that avgpool layer shows 2048. However, when I try this solution from @ptrblck it doesn’t recognize the syntax.

      (branch3x3dbl_3b): BasicConv2d(
        (conv): Conv2d(384, 384, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0), bias=False)
        (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)
      )
      (branch_pool): BasicConv2d(
        (conv): Conv2d(2048, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
    (dropout): Dropout(p=0.5, inplace=False)
    (fc): Linear(in_features=2048, out_features=2, bias=True)
  )
)>

I tried:
model.avgpool.register_forward_hook(get_activation('avgpool'))

How can l load my best model as a feature extractor/evaluator?

I’ve created a small code snippet using a forward hook to store one activation from fc2: class MyModel(nn.Module): def __init__(self): super(MyModel, self).__init__() self.cl1 = nn.Linear(25, 60) self.cl2 = nn.Linear(60, 16) self.fc1 = nn.Linear(16, 120) self.fc2 = nn.Linear(120, 84) self.fc3 = nn.Linear(84, 10) def forward(self, x): x = F.relu(self.cl1(x)) x = F.relu(self.cl2(x)) x = F.relu(self.fc1(x)) …

[ptrblck]

The error you are seeing is unrelated to PyTorch and most likely caused by Jupyter Notebook Renderers Extension as described e.g. here.

[Mona_Jalal]

Thank you for pointing me to that link. However, it also seems that get_activation is not recognized. Is the code in cell #12 correct for inception v3 in your opinion?

[ptrblck]

Yes it looks correct and also works fine:

activation = {}
def get_activation(name):
    def hook(model, input, output):
        activation[name] = output.detach()
    return hook

model = models.inception_v3()
model.avgpool.register_forward_hook(get_activation("avgpool"))

x = torch.randn(2, 3, 299, 299)
out = model(x)

print(activation['avgpool'].shape)
# torch.Size([2, 2048, 1, 1])

[Mona_Jalal]

you were correct that the specific renderer error was on VSCode Jupyter notebook side. I had the extension installed but I uninstalled it and installed it back again and the problem is gone.

^ this is the version I had initially. I also uninstalled and installed the Jupyter extension.

Here’s the error I am getting:

AttributeError: ‘DataParallel’ object has no attribute ‘avgpool’

From the link you shared, I used this answer: python - Error loading preloads: Could not find renderer - Stack Overflow

[Mona_Jalal]

I get an error using your exact code copied:

> AttributeError: ‘DataParallel’ object has no attribute ‘avgpool’

[Mona_Jalal]

fixed it using module as suggested by you here

Fine tuning resnet: 'DataParallel' object has no attribute 'fc'

When using DataParallel your nn.Module will be in .module: model = Net() model.fc model_parallel = nn.DataParallel(model) model_parallel.module.fc