Use pretrained resnet18 for segmentation

Cverlpeng · April 9, 2019, 5:40am

hi,everyone
I rebuild resnet18 and use pretrained of pytorch for segmentation task, I trained this model,but the network has not learned anything.Is this written correctly? Is there a more concise way of writing? I not use average pool and fc layers of resnet18 ,I want use pretrained weight of pytorch !

thanks !!!

class ResNet18(nn.Module):
    def __init__(self, out_dim, *args, **kwargs):
        super(ResNet18, self).__init__(*args, **kwargs)
        resenet18 = torchvision.models.resnet18(pretrained=True)
        self.conv1 = resenet18.conv1 
        self.bn1 = resenet18.bn1
        self.relu = resenet18.relu
        self.maxpool = resenet18.maxpool
        self.c1 = resenet18.layer1
        self.c2 = resenet18.layer2
        self.c3 = resenet18.layer3
        self.c4 = resenet18.layer4
        self.deconv = nn.ConvTranspose2d(512, out_dim, kernel_size=32, stride=32, padding=0, output_padding=0)

    def forward(self, x):
        x =self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)
        x = self.c1(x)
        x = self.c2(x)
        x = self.c3(x)
        x = self.c4(x)        
        x = self.deconv(x)
        x = [x]
        return tuple(x)

Cverlpeng · April 9, 2019, 12:56pm

who can help me ? thanks

ptrblck · April 10, 2019, 8:17am

The model looks generally alright. I’m wondering a bit why you are wrapping x in a list and return a tuple?
Could you try to overfit the model on a very small sample of your dataset, e.g. just 10 examples?
If that’s not working you might have some other bugs in your code.

Cverlpeng · April 10, 2019, 9:28am

I am imitating this file DANet
I return tuple due to

Current PyTorch DataParallel Table is not supporting mutl-gpu loss calculation, which makes the gpu memory usage very in-balance. We address this issue here by doing DataParallel for Model & Criterion. Parallel

I trained this model on lane segmentation dataset , it has train set 1319，I get this resulit :

this image have four images,The bottom right corner of the picture is network input lane label image ,other three images is softmax result of resnet18 model output 。( result_softmax * 255 )

very faster when i trained model. print training log some abnormalities, some times print two epoch log,some times print one epoch informention。

this train model learning rate and loss

I only change network, used rebuild resnet18,but when I use deeplabv1 model,it can segmentaion lane image

This problem has been bothering me for a few days. I hope you can help me, thank you very much!

ptrblck · April 10, 2019, 9:53am

Did you try to overfit a small data sample?
I’m not familiar with deeplabv1, but did you only change the model or something else?

Cverlpeng · April 10, 2019, 10:55am

I not try to overfit a small data sample, yes ,I only change network Network structure, The rest of the steps are not changed, when i use deeplabv1 ( Code reproduced by others ) to training, it can segmentation image. but I use resnet18 (Code reproduced by me) ,it not work, so I think reproduced code have error !

Cverlpeng · April 10, 2019, 11:03am

I am pytorch novice, I don’t quite understand why, many open split jobs will rewrite DataParallel,such as parallel, but don’t use official this function.

ptrblck · April 10, 2019, 8:30pm

Ok, but if your model succesfully overfits on a small sample, this would be at least some signal that your code is working, so I would suggest to try it first.

I’m not familiar with the package you’ve linked, but apparently it’s used to avoid an imbalances GPU memory usage in nn.DataParallel.

Cverlpeng · April 11, 2019, 1:29am

Thank you for your prompt reply
I rebuild vgg16 ,it can work(pretrained = True or False) , I rebulit resnet18, it can’t work, but I don know why?

class VGG16(nn.Module):
    def __init__(self, out_dim, *args, **kwargs):
        super(VGG16, self).__init__(*args, **kwargs)
        vgg16 = torchvision.models.vgg16(pretrained=True)
        self._up_kwargs = {'mode': 'bilinear', 'align_corners': True}
        self.features = vgg16.features
        self.conv1x1 = nn.Conv2d(512, out_dim, 1,1,0)
        self.deconv = nn.ConvTranspose2d(512, out_dim, kernel_size=32, stride=32, padding=0, output_padding=0)
    def forward(self, x):
        imsize = x.size()[2:]
        x =self.features(x)
        x = self.conv1x1(x)
        x = F.upsample(x, imsize, **self._up_kwargs)
        x = [x]
        return tuple(x)

class ResNet18(nn.Module):
    def __init__(self, out_dim, *args, **kwargs):
        super(ResNet18, self).__init__(*args, **kwargs)
        resenet18 = torchvision.models.resnet18(pretrained=True)
        self.conv1 = resenet18.conv1 
        self.bn1 = resenet18.bn1
        self.relu = resenet18.relu
        self.maxpool = resenet18.maxpool
        self.c1 = resenet18.layer1
        self.c2 = resenet18.layer2
        self.c3 = resenet18.layer3
        self.c4 = resenet18.layer4
        self.deconv = nn.ConvTranspose2d(512, out_dim, kernel_size=32, stride=32, padding=0, output_padding=0)

    def forward(self, x):
        x =self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)
        x = self.c1(x)
        x = self.c2(x)
        x = self.c3(x)
        x = self.c4(x)        
        x = self.deconv(x)
        x = [x]
        return tuple(x)

I find resulit of segmentation,The jagged phenomenon is obvious. I try to use Multiple models , but get same result. A small square appears in the lane probability map.

this image show result of Vgg16

amk · December 9, 2019, 11:57am

I encountered the same issue. Have you had any success meanwhile?