Feature exctraction resent50 using nn.Sequential

vision
1

Hello, I want to use resnet50 on my own data set and extract features after training on my data. After I am done with the training, I want to save the features before softmax layer. I have used nn.Sequential to add some layers to the resenet50 model, but I cannot access inside the Sequential. Any help is much appreciated. My code looks like this:

model = models.resnet50(pretrained=True)
n_inputs = model.fc.in_features
model.fc = nn.Sequential(
nn.Linear(n_inputs, 256),
nn.ReLU(),
nn.Dropout(0.4),
nn.Linear(256, n_classes),
nn.LogSoftmax(dim=1))

2

Hi Negin, you can access inside the Sequential just by indexing, e.g.:

n_inputs = 1000
n_classes = 5

fc = nn.Sequential(
nn.Linear(n_inputs, 256),
nn.ReLU(),
nn.Dropout(0.4),
nn.Linear(256, n_classes),
nn.LogSoftmax(dim=1))

then:

fc[0]
Out: Linear(in_features=1000, out_features=256, bias=True)

fc[0].weight
Out: Parameter containing:
tensor([[-0.0103,  0.0170,  0.0160,  ...,  0.0275, -0.0294,  0.0309],
        [ 0.0046, -0.0301, -0.0032,  ...,  0.0195, -0.0224, -0.0095],
        [ 0.0229, -0.0076, -0.0062,  ..., -0.0256, -0.0065,  0.0039],
        ...,
        [ 0.0161, -0.0019,  0.0032,  ..., -0.0299,  0.0106,  0.0189],
        [-0.0208,  0.0051, -0.0269,  ..., -0.0016,  0.0144,  0.0314],
        [-0.0121,  0.0188, -0.0211,  ..., -0.0263,  0.0120,  0.0235]],
       requires_grad=True)

fc[0].bias
Out: Parameter containing:
tensor([ 0.0077, -0.0094, -0.0161,  0.0084,  0.0064,  0.0057, -0.0187, -0.0269,
        -0.0020, -0.0084, -0.0150,  0.0281,  0.0314,  0.0107, -0.0049,  0.0230,
        -0.0002, -0.0055,  0.0282, -0.0007,  0.0032, -0.0141, -0.0316, -0.0212,
        -0.0295,  0.0166, -0.0097,  0.0305, -0.0060,  0.0247,  0.0150, -0.0206,
        -0.0310, -0.0130, -0.0203,  0.0024, -0.0096, -0.0056, -0.0157,  0.0237,
        -0.0023, -0.0076,  0.0262,  0.0035, -0.0052, -0.0117, -0.0136, -0.0174,
        -0.0239,  0.0145, -0.0225, -0.0170,  0.0268, -0.0238, -0.0289,  0.0060,
         0.0129, -0.0061,  0.0314, -0.0150,  0.0237,  0.0014, -0.0137, -0.0300,
        -0.0041,  0.0194,  0.0170,  0.0037, -0.0310,  0.0257, -0.0073, -0.0109,
         0.0289, -0.0143,  0.0208,  0.0264,  0.0072, -0.0010,  0.0068, -0.0172,
         0.0095,  0.0193,  0.0135, -0.0153,  0.0310,  0.0064,  0.0116,  0.0136,
        -0.0186,  0.0112, -0.0133,  0.0048,  0.0233, -0.0048, -0.0292, -0.0122,
        -0.0215,  0.0241,  0.0043,  0.0152, -0.0042,  0.0024,  0.0279, -0.0169,
         0.0155, -0.0184,  0.0272,  0.0266,  0.0069,  0.0277, -0.0247,  0.0301,
        -0.0241,  0.0148, -0.0294, -0.0063,  0.0074, -0.0133,  0.0177,  0.0173,
        -0.0070, -0.0056,  0.0139,  0.0121,  0.0222, -0.0146, -0.0179,  0.0232,
         0.0108, -0.0156, -0.0280, -0.0249, -0.0123,  0.0237,  0.0043, -0.0047,
         0.0007,  0.0071,  0.0222,  0.0025,  0.0221, -0.0106, -0.0098, -0.0314,
         0.0267,  0.0116, -0.0074, -0.0107,  0.0028, -0.0200,  0.0099, -0.0246,
         0.0149,  0.0159, -0.0142, -0.0026,  0.0077,  0.0040, -0.0191,  0.0178,
        -0.0179, -0.0225,  0.0103,  0.0185, -0.0024,  0.0027, -0.0281,  0.0069,
        -0.0132, -0.0046, -0.0024,  0.0175,  0.0265,  0.0006,  0.0010, -0.0278,
         0.0204,  0.0226, -0.0247,  0.0204, -0.0003, -0.0005, -0.0255, -0.0023,
         0.0024,  0.0022, -0.0065,  0.0257,  0.0014,  0.0218, -0.0029, -0.0126,
         0.0278, -0.0309,  0.0044, -0.0001,  0.0115,  0.0032, -0.0184,  0.0239,
         0.0240,  0.0016,  0.0297,  0.0233,  0.0222, -0.0126, -0.0221,  0.0220,
         0.0262,  0.0286, -0.0129,  0.0275, -0.0284, -0.0280,  0.0042, -0.0126,
        -0.0196, -0.0058, -0.0083, -0.0178, -0.0133,  0.0259, -0.0164, -0.0141,
         0.0167, -0.0045,  0.0071,  0.0190,  0.0057, -0.0078,  0.0194, -0.0040,
        -0.0260,  0.0125,  0.0036, -0.0144, -0.0133,  0.0276, -0.0221,  0.0226,
        -0.0264,  0.0144,  0.0190, -0.0275,  0.0238, -0.0014, -0.0134, -0.0257,
        -0.0118,  0.0083,  0.0248,  0.0253,  0.0226, -0.0315, -0.0301,  0.0250],
       requires_grad=True)

Also, this link explains how to save to disk.

3

@Andrei_Cristea Thank you very much for your reply. I want to input some new images (based on my trained model) and save the features of the new images that I input to the trained model. I want to save the model in a way that it output the features of model.fc[3]
If I go with indexing:

1
1872×453 23.2 KB

to return the list to the format of resnet model. I do not know what to do. I have tried Sequential
2
2824×553 22 KB

but it does not work.
Basically I want to pass mu images to the model but instead of getting 5 5 features which are the number of classes. I want to get an intermediate layer feature on my sequential.

5

Hi negin,

Is this what you’re looking for?

import torch, torchvision
from PIL import Image

model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18', pretrained=True)

model.fc = torch.nn.Sequential(  # replacing fc layer with Sequential
    torch.nn.Linear(512, 10),  # fc[0]
    torch.nn.Linear(10, 5),    # fc[1]
    torch.nn.Linear(5, 3),     # fc[2]
    torch.nn.Linear(3, 10),    # fc[3]
)

img = Image.new('RGB', (224, 224))  # mock input image
input_tensor = torchvision.transforms.ToTensor()(img)  # convert to tensor first
output = model(input_tensor.unsqueeze(0))  # add bogus "batch" dimension
output_to_save = output.detach().cpu().numpy()  # convert to numpy

np.save("output.npy", output_to_save)  # save it like this
print(output_to_save)

Output:
[[ 0.27786672 -0.35223806  0.46923774 -0.37864536  0.11765398  0.32065433
  -0.23159212 -0.20262967 -0.16044387  0.689993  ]]

Obviously adapted to your needs.

6

Hi Andrei,

No, I want to train my model first. if I train my model my model.fc works best like this:

model = models.resnet50(pretrained=True)

model.fc = nn.Sequential(
                      nn.Linear(n_inputs, 256), 
                      nn.ReLU(), 
                      nn.Dropout(0.4),
                      nn.Linear(256, 128),
                      nn.Linear(128, n_classes),                   
                      nn.LogSoftmax(dim=1))

After I am done with training, I want to input some images to my model and it returns to me 128 features before my n_classes which are only 5 features.
If I would have 10 features in the last layer or 128 layers. I cannot train my model based on the 5 classes I have

8

Understood! You can do this:

import torch, torchvision
from torch import nn
from PIL import Image

model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18', pretrained=True)
n_inputs = 512
n_classes = 5

model.fc = torch.nn.Sequential(  # Suppose this is your sequential model
    nn.Linear(n_inputs, 256),   # fc[0]
    nn.ReLU(),                  # fc[1]
    nn.Dropout(0.4),            # fc[2]
    nn.Linear(256, 128),        # fc[3]
    nn.Linear(128, n_classes),  # fc[4] 
)

# the above ends with n_classes=5, which you don't want
# instead, you want to end with the layer outputting 128
# so let's replace the Sequential with one 
# that doesn't have the last layer
# but has all the prior layers

model.fc = nn.Sequential(
    model.fc[0],
    model.fc[1],
    model.fc[2],
    model.fc[3],
)

# let's test if this indeed returns 128 'classes'
img = Image.new('RGB', (224, 224))  # input image
input_tensor = torchvision.transforms.ToTensor()(img)  # convert to tensor first
output = model(input_tensor.unsqueeze(0))  # add bogus "batch" dimension
output_to_save = output.detach().cpu().numpy()  # convert to numpy

print(output_to_save)
np.save("output.npy", output_to_save)  # save it like this

Output:
[[ 0.01438853  0.87196857 -0.2514455  -0.04714714  0.07090778 -0.09341301
  -0.2072835  -0.3704936  -0.01389392  0.18359633  0.10641675 -0.11172692
   0.6255953  -0.04955412 -0.05233074 -0.23172098 -0.64670634  0.01767935
  -0.39153028  0.09038768 -0.4336232   0.5385093   0.09623256  0.05085237
   0.141657    0.10092676 -0.35657144 -0.31972992  0.8487682  -0.10586535
   0.02011631  0.34999335 -0.07444657 -0.06364524  0.68986833 -0.26194543
  -0.10996548 -0.4478173   0.08224907 -0.22302929  0.08527138  0.34809947
   0.31143022  0.4006711   0.06359468  0.2257615   0.42476267  0.30963692
   0.12626037 -0.25697252  0.16672882 -0.06121482 -0.00894236 -0.13477468
  -0.28371924 -0.5638889   0.23877671 -0.55630106 -0.19719738  0.15854624
   0.07817743 -0.3680797   0.21859777  0.01053216  0.4597964  -0.37222108
  -0.1515545   0.13200204 -0.34337598 -0.09385173  0.2707774   0.18420342
  -0.14283875  0.3390723  -0.25316817 -0.22421657 -0.7452298   0.08496317
   0.07830979 -0.06452103 -0.19457048 -0.10110219  0.19297805  0.35950035
  -0.3854925   0.19778475  0.12232781 -0.04913102  0.40340108  0.6487622
   0.03393503  0.31627747  0.4140402  -0.6647738  -0.45569345 -0.51362383
  -0.8327177   0.0949322  -0.0045674  -0.23051375 -0.11645541 -0.45121124
  -0.01439564 -0.0192363  -0.22718215 -0.00993027  0.09125008 -0.13285461
  -0.20360185  0.47607702  0.11188114  0.1484635   0.18721539 -0.36961102
  -0.09044638 -0.2350381   0.41334015 -0.2741512  -0.39748335  0.15688536
   0.11036351 -0.60628456  0.3990275  -0.340592    0.5213363  -0.06208746
   0.34663257 -0.08785937]]  # 128 elements here!

Also see this discussion, where a similar question is answered.

Hope this helps!

9

@Andrei_Cristea Thank you so much for taking time. This is exactly what I wanted and I was looking for an answer everywhere. Thank you! It works.