i’m sorry about the week language.
I want to use the DeepLabV3+_resnet101 model pretrained, and to add some new connection between layers.
for example, in the first step I want to apply a new conv2d layer on the input layer and the last conv layer in the model.
I tried to looking for examples of transfer learning, but all what I find is examples of changing just the last FC layer…
I tried to take the layers form the model, and reconstuct them with some new layers, such in the following section (it’s not the model I wanted to build, it’s just a test).
import math
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.utils.model_zoo as model_zoo
import torchvision.models as models
class deeplab_v3_separation(nn.Module):
def __init__(self):
super(deeplab_v3_separation, self).__init__()
deepLabV3ResNet101 = models.segmentation.deeplabv3_resnet101(pretrained=True, progress=True)
features_convs_and_ASSP = list(deepLabV3ResNet101.modules())
self.features_convs_and_ASSP = torch.nn.Sequential(*features_convs_and_ASSP[:])
self.conv1 = nn.Conv2d(3, 21, kernel_size=1, bias=False)
self.conv2 = nn.Conv2d(21, 5, kernel_size=1, bias=False)
self.bn1 = nn.BatchNorm2d(5)
def forward(self, x):
x = self.features_convs_and_ASSP(x)
x = self.conv1(x)
x = self.conv2(x)
x = self.bn1(x)
return x
I got some errors about the types of the “tensor” that flow over the layers.
Finaly I had understood that this model have two types of outputs and I should choose the one i wanted, and when I choose the one I wanted the entire model works well.
On the other hand, I did some steps since opening this topic but I still got some prolems with building the exact model I wanted:
I should separate between the resnet101 and the deepLbv3, and when I try to do so I got a problem that the deeplabv3’s output size is too small, instead of the original image size.
I tryed to take the resnet101 by itself, and to reconstruct only the deeplabv3, and it seems that the sepapartion of the deeplabv3 model to children, and recombining them together changing the model.
this is the py file I tryed:
import torch
import urllib
from PIL import Image
from torchvision import transforms
import matplotlib.pyplot as plt
from model import deeplab_v3_separation
import torchvision.models as models
deepLabV3ResNet101 = models.segmentation.deeplabv3_resnet101(pretrained=True)
# features_convs_and_ASSP = list(deepLabV3ResNet101.children())
# deepLabV3ResNet101_test = torch.nn.Sequential(*features_convs_and_ASSP)
# deepLabV3ResNet101_test.eval()
deepLabV3ResNet101.eval()
ResNet101 = models.resnet101(pretrained=True)
ResNet101_features = list(ResNet101.children())
ResNet101_test = torch.nn.Sequential(*ResNet101_features[:-2])
ResNet101_test.eval()
ASSP_layers = models.segmentation.deeplabv3_resnet101(pretrained=True)
ASSP_features = list(ASSP_layers.children())
# ASSP_classifier = list(ASSP_layers.classifier.children())
ASSP_test = torch.nn.Sequential(*ASSP_features[1:2])
ASSP_test.eval()
input_image = Image.open('a.jpg')
preprocess = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(input_image)
input_batch = input_tensor.unsqueeze(0) # create a mini-batch as expected by the model
# move the input and model to GPU for speed if available
if torch.cuda.is_available():
input_batch = input_batch.to('cuda')
ResNet101_test.to('cuda')
ASSP_test.to('cuda')
deepLabV3ResNet101.to('cuda')
with torch.no_grad():
output = ResNet101_test(input_batch)
output2 = ASSP_test(output)
output3 = deepLabV3ResNet101(input_batch)
print(output.size())
print(output2.size())
print(output3['out'].size())
I think that’s a valid concern and I would not recommend to blindly pass all child modules to an nn.Sequential container.
This approach might work fine for simple models, which define and call all modules in a sequential order, but will break for every other use case.
If you want to use this approach, I would recommend to check the original model implementation with the nn.Sequential container and make sure all wanted modules are in the right order.
Alternatively, you can derive a custom class using the base model and redefine the forward as you wish.