How do I load and use an already retrained InceptionV3 classifier?

I’m very new to PyTorch - I typically use other libraries for neural networks.

However, I have a model that I found which is a retrained InceptionV3 classifier, which was retrained using PyTorch. I don’t want to have to retrain the classifier myself in a library I’m familiar with.

The model I’d like to load and use is saved as a .pth.tar file. I guess I’ve figured out how to load the model:
model = torch.load('iNat_2018_InceptionV3.pth.tar', map_location='cpu')
This seems to work, because when I print model, it prints out a ton of numbers, which look like values for weights, biases, pooling layers, etc.

Now that I’ve loaded the model (hopefully correctly, please correct me if I’ve made a mistake with loading it), I’d like to use it to classify images. This is the part I’m completely clueless about. First of all, how must I format and provide the input? Should I convert the image into an array? Should I format the image in any way? When that’s done, how do I pass the input to the neural network to classify? How can I get the outputs?

Thank you for your help.

Could you post the repo you’ve found this model?
If the pront statement just shows a lot of numbers, you probably loaded the state_dict, not the model itself.
Have a look at the Serialization semantics for more information about how to restore the model using a state_dict.

Basically you should copy the model definition (derived from nn.Module), load the state_dict, set to model.eval(), and pass your input to it.

Your input has to be a torch.tensor. Have a look at this beginner tutorial.

If you load an image using PIL, you can use torchvision.transforms.ToTensor() to get a scaled tensor.
Your input should have the shape [batch_size, channels, height, width].

Depending on your model, the spatial size might be fixed, e.g. to 224.

1 Like

Thanks for the help!

Here’s the link to the repo I found the model on:

It looks like they used the same Inception model code from torchvision, but changed the pooling operation to an adaptive pooling. This would mean you can pass variable sized input into your model.
Could you try the following:

model = inception_v3(pretrained=False)
model.fc = nn.Linear(2048, args.num_classes)
model.aux_logits = False
model.load_state_dict(torch.load('iNat_2018_InceptionV3.pth.tar', map_location='cpu'))
model.eval()

x = torch.randn(1, 3, 299, 299) # dummy image
output = model(x)
1 Like

I tried out the following code:

import os
import shutil
import time
import numpy as np

import torch
import torch.nn as nn
import torch.optim
import torch.utils.data

import train_inat
from inception import *
import inat2018_loader

model = inception_v3(pretrained=False)
model.fc = nn.Linear(2048, args.num_classes)
model.aux_logits = False
model.load_state_dict(torch.load('iNat_2018_InceptionV3.pth.tar', map_location='cpu'))
model.eval()

x = torch.randn(1, 3, 299, 299) # dummy image
output = model(x)

It produces the following error:

  File "test.py", line 16, in <module>
    model.fc = nn.Linear(2048, args.num_classes)
NameError: name 'args' is not defined

I also removed the following code from train_inat.py because I didn’t want main() to run, which’ll train the network:

if __name__ == '__main__':
    main()

You can see these arguments in their training code.
In train_inat.py they used num_classes = 8142.

1 Like

Okay, because they’ve declared args within train_inat.py, I’ve replaced the main() function (which trains the network) in train_inat.py with this:

def main():
    global args, best_prec3
    args = Params()
    
    model = inception_v3(pretrained=False)
    model.fc = nn.Linear(2048, args.num_classes)
    model.aux_logits = False
    model.load_state_dict(torch.load('iNat_2018_InceptionV3.pth.tar', map_location='cpu'))
    model.eval()

    x = torch.randn(1, 3, 299, 299) # dummy image
    output = model(x)

I’m not sure if this is the way to do it, or if I’ve removed some required code.

I get the following error when I run this modified train_inat.py file:

RuntimeError: Error(s) in loading state_dict for Inception3:
	Missing key(s) in state_dict:

This error was followed by a ton of text.

I think the easiest way would be to set your checkpoint path in this line and set evaluate=True in this line, and run their script.

1 Like

So here’s my new Params class:

class Params:
    # arch = 'inception_v3'
    num_classes = 8142
    workers = 6
    epochs = 100
    start_epoch = 0
    batch_size = 64  # might want to make smaller
    lr = 0.0045
    lr_decay = 0.94
    epoch_decay = 4
    momentum = 0.9
    weight_decay = 1e-4
    print_freq = 100

    resume = 'iNat_2018_InceptionV3.pth.tar'  # set this to path of model to resume training
    #resume = '/home/macaodha/Projects/inat2018/code/model_best.pth.tar'
    train_file = '/media/Storage_1/Datasets/inat2018/train2018.json'
    val_file = '/media/Storage_1/Datasets/inat2018/val2018.json'
    data_root = '/media/Storage_1/Datasets/inat2018/images/'

    # set evaluate to True to run the test set
    evaluate = True
    save_preds = True
    op_file_name = 'inat2018_test_preds.csv' # submission file
    if evaluate == True:
        val_file = '/media/Storage_1/Datasets/inat2018/test2018.json'

I didn’t change any other code besides the main() function as you saw above.

When I run the script, it still displays the same error as previously mentioned.

I guess it’s because the state_dict is saved in the checkpoint dict:

if os.path.isfile(args.resume):
    print("=> loading checkpoint '{}'".format(args.resume))
    checkpoint = torch.load(args.resume)
    args.start_epoch = checkpoint['epoch']
    best_prec3 = checkpoint['best_prec3']
    model.load_state_dict(checkpoint['state_dict'])
    optimizer.load_state_dict(checkpoint['optimizer'])
    print("=> loaded checkpoint '{}' (epoch {})"
        .format(args.resume, checkpoint['epoch']))

I would try to stick to their code as much as possible, i.e. use their load functions etc.

1 Like

Should I add that to the main() function or to the Params class?

To the main function, where you are restoring your model.

1 Like

Here’s my new main() function:

def main():
    global args, best_prec3
    args = Params()

    model = inception_v3(pretrained=False)

    if os.path.isfile(args.resume):
        print("=> loading checkpoint '{}'".format(args.resume))
        checkpoint = torch.load(args.resume, map_location='cpu')
        args.start_epoch = checkpoint['epoch']
        best_prec3 = checkpoint['best_prec3']
        model.load_state_dict(checkpoint['state_dict'])
        optimizer.load_state_dict(checkpoint['optimizer'])
        print("=> loaded checkpoint '{}' (epoch {})"
            .format(args.resume, checkpoint['epoch']))

    model.fc = nn.Linear(2048, args.num_classes)
    model.aux_logits = False
    model.load_state_dict(torch.load('iNat_2018_InceptionV3.pth.tar', map_location='cpu'))
    model.eval()

    x = torch.randn(1, 3, 299, 299) # dummy image
    output = model(x)

I’ve used the above main() function, which produced the following:

RuntimeError: Error(s) in loading state_dict for Inception3:
	size mismatch for fc.weight: copying a param of torch.Size([1000, 2048]) from checkpoint, where the shape is torch.Size([8142, 2048]) in current model.
	size mismatch for fc.bias: copying a param of torch.Size([1000]) from checkpoint, where the shape is torch.Size([8142]) in current model.

Still can’t seem to figure it out. Any possible solutions? Thanks.

How many classes do you have?
I’m not familiar with the repo and its properties, but it seems the checkpoint uses 1000 classes, while args.num_classes=8142 by default. Try to change it to 1000.

1 Like

I’ve tried modifying num_classes in Params class to 1000, yet it still throws the same error.

Any ideas on how to fix this?

Is it exactly the same error or do you get another size mismatch?

1 Like

The exact same error.

main():

def main():
    global args, best_prec3
    args = Params()

    model = inception_v3(pretrained=False)

    if os.path.isfile(args.resume):
        print("=> loading checkpoint '{}'".format(args.resume))
        checkpoint = torch.load(args.resume, map_location='cpu')
        args.start_epoch = checkpoint['epoch']
        best_prec3 = checkpoint['best_prec3']
        model.load_state_dict(checkpoint['state_dict'])
        optimizer.load_state_dict(checkpoint['optimizer'])
        print("=> loaded checkpoint '{}' (epoch {})"
            .format(args.resume, checkpoint['epoch']))

    model.fc = nn.Linear(2048, args.num_classes)
    model.aux_logits = False
    model.load_state_dict(torch.load('iNat_2018_InceptionV3.pth.tar', map_location='cpu'))
    model.eval()

    x = torch.randn(1, 3, 299, 299) # dummy image
    output = model(x)

Params class:

class Params:
    # arch = 'inception_v3'
    num_classes = 1000
    workers = 6
    epochs = 100
    start_epoch = 0
    batch_size = 64  # might want to make smaller
    lr = 0.0045
    lr_decay = 0.94
    epoch_decay = 4
    momentum = 0.9
    weight_decay = 1e-4
    print_freq = 100

    resume = 'iNat_2018_InceptionV3.pth.tar'  # set this to path of model to resume training
    #resume = '/home/macaodha/Projects/inat2018/code/model_best.pth.tar'
    train_file = '/media/Storage_1/Datasets/inat2018/train2018.json'
    val_file = '/media/Storage_1/Datasets/inat2018/val2018.json'
    data_root = '/media/Storage_1/Datasets/inat2018/images/'

    # set evaluate to True to run the test set
    evaluate = True
    save_preds = True
    op_file_name = 'inat2018_test_preds.csv' # submission file
    if evaluate == True:
        val_file = '/media/Storage_1/Datasets/inat2018/test2018.json'

Error:

=> loading checkpoint 'iNat_2018_InceptionV3.pth.tar'
Traceback (most recent call last):
  File "train_inat.py", line 230, in <module>
    main()
  File "train_inat.py", line 57, in main
    model.load_state_dict(checkpoint['state_dict'])
  File "/home/aditya/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 719, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Inception3:
	size mismatch for fc.weight: copying a param of torch.Size([1000, 2048]) from checkpoint, where the shape is torch.Size([8142, 2048]) in current model.
	size mismatch for fc.bias: copying a param of torch.Size([1000]) from checkpoint, where the shape is torch.Size([8142]) in current model.

I tried to load the state_dict from the checkpoint and it seems the number of classes was 8142.
This code snippet works on my machine:

model = models.inception_v3(pretrained=False)
model.fc = nn.Linear(2048, 8142)
model.aux_logits = False
model = model.to('cuda')

checkpoint = torch.load('./iNat_2018_InceptionV3.pth.tar')
state_dict = checkpoint['state_dict']
model.load_state_dict(state_dict)
1 Like