Hi All,
How can I modify the deeplabv3_resnet101 and fcn_resnet101 models available from torchvision segmentation models to accept input images with only 1 color channel?
I have seen some example of how I can modify resnet, but I am not sure how to do it for these
Thanks
Nishanth
You could replace the first conv layer in a similar manner as done with resnet:
model = torchvision.models.segmentation.deeplabv3_resnet101()
model.backbone.conv1 = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
x = torch.randn(2, 1, 224, 224)
output = model(x)
hello
I faced the same issue, I have tried to fine-tune deeplabv3_resnet50/resnet101 and fcn_resnet50/resnet101 on grayscale images(one channel as input). I tried your solution mentioned above but it’s not working, this is the code that I m using
class FCN_resnet50(nn.Module):
def init(self, num_classes=2, pretrained=False):
super(FCN_resnet50, self).init()
model=fcn_resnet50(pretrained)
# model.eval()
# model=timm.create_model(‘fcn_resnet50’, pretrained, in_chans=1)
# take pretrained resnet, except AvgPool and FC
# self.conv_weight = model.conv1[0].weight
# # nn.Conv2d(1, self.inplanes, kernel_size=3, padding=1,
# # bias=False)
# model.conv1[0].in_channels=1
# model.conv1[0].weight=torch.nn.parameter(self.conv_weight.sum(dim=1, keepdim=True))
# weight1 = model.features.conv1.weight.clone()
# new_first_layer = nn.Conv2d(1, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3),
# bias=False).requires_grad_()
# new_first_layer.weight[:, :3, :, :].data[...] = Variable(weight1, requires_grad=True)
# model.features.conv1 = new_first_layer
model.backbone.conv1 = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
self.conv1=model.backbone.conv1
self.bn1 = model.bn1
self.relu = model.relu
self.maxpool = model.maxpool
self.layer1 = model.layer1
self.layer2 = model.layer2
self.layer3 = model.layer3
self.layer4 = model.layer4
self.last_conv = nn.Conv2d(2048, num_classes, kernel_size=1)
As you see I tried multiple instructions but no one worked for me.
I will appreciate your help.
many thanks.
Your code is unfortunately not formatted correctly and also not executable.
What kind of issue are you seeing using my code snippet?
RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[16, 1, 384, 384] to have 3 channels, but got 1 channels instead
I just executed my code snippet again and don’t see the error. Are you sure you are running this code?
yes I m sure that using this model, my aim is to modify deeplabv3_resnet50/resnet101 and fcn_resnet50/resnet101 to segment medical imaging that is stored in 2d grayscale images
then I load the model through pytorch lightning module.
I meant to ask if you were executing my posted code snippet directly as I do not see the mentioned error. If you manipulate my code or are using it in another way, your change might cause the issue in which case a minimal and executable code snippet would be needed to help debugging it.
