Multiple Output from MobileNet V3

vision
1

Hi,

Following the approaches seen, I tried to add a second output to mobilenet_v3 using the following:

class MBN_v3(torch.nn.Module):
    def __init__(self, num_classes1, num_classes2):
        super(MBN_v3, self).__init__()
        self.model_resnet = mobilenet_v3_small(pretrained=True)
        
        self.model_resnet.fc = torch.nn.Identity()
        self.classifier1 = torch.nn.Linear(1024, num_classes1)
        self.classifier1 = torch.nn.Linear(1024, num_classes2)

    def forward(self, x):
        x = self.model_resnet(x)
        out1 = self.classifier1(x)
        out2 = self.classifier2(x)
        
        return out1, out2

The self.classifer1 tagging seem to the one causing an error but I am not sure how to interpret the following output (with some omissions):

 ** On entry to SGEMM  parameter number 10 had an illegal value
Traceback ...
packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs)
 ... 
line 1051, in _call_impl return forward_call(*input, **kwargs) return F.linear(input, self.weight, self.bias)
... 
line 1847, in linear return torch._C._nn.linear(input, weight, bias)
RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

Any insight or suggestion would be highly appreciated!

P

2

Hi P,

I replicated your code and I don’t have the same error. There are a few comments for you though:

  1. pretrain is deprecated, you can use the definition in the code below;
  2. classifier2 is not defined, you have a copy-paste error in the constructor;
  3. the output of model_resnet has 1000 classes instead of 1024, maybe this is also the source of your error but I got a simple mismatch error instead.
In [1]: from torchvision.models import mobilenet_v3_small, MobileNet_V3_Small_Weights

In [2]: import torch

In [3]: class MBN_v3(torch.nn.Module):
   ...:     def __init__(self, num_classes1, num_classes2):
   ...:         super(MBN_v3, self).__init__()
   ...:         self.model_resnet = mobilenet_v3_small(weights=MobileNet_V3_Small_Weights.DEFAULT)
   ...: 
   ...:         self.model_resnet.fc = torch.nn.Identity()
   ...:         self.classifier1 = torch.nn.Linear(1000, num_classes1)
   ...:         self.classifier2 = torch.nn.Linear(1000, num_classes2)
   ...: 
   ...:     def forward(self, x):
   ...:         x = self.model_resnet(x)
   ...:         out1 = self.classifier1(x)
   ...:         out2 = self.classifier2(x)
   ...: 
   ...:         return out1, out2
   ...: 

In [4]: model = MBN_v3(10, 20)

In [5]: inp = torch.randint(0, 256, (1, 3, 256, 256), dtype= torch.float32)

In [6]: model(inp)
Out[6]: 
(tensor([[-3.8697, -0.3494,  0.3550, -0.3996,  0.1518,  1.0698, -1.8423,  2.0341,
          -2.8451, -3.5946]], grad_fn=<AddmmBackward0>),
 tensor([[ 2.7698,  0.6458,  1.0684, -0.1247,  1.7675, -1.9195,  0.7464, -2.3890,
          -0.9420, -0.1669,  2.5537, -0.3263, -1.5426,  1.0682,  0.8419, -1.6700,
           0.9832, -1.5306,  0.0394, -1.7743]], grad_fn=<AddmmBackward0>))
3

Thank you Gian!

The class number of 1024 was causing the error indeed. Do the modified classifiers actually become an additional layer added to the original classifier as printed below? I want to make sure that I understand where the 1000 is coming from.

  (classifier): Sequential(
    (0): Linear(in_features=576, out_features=1024, bias=True)
    (1): Hardswish()
    (2): Dropout(p=0.2, inplace=True)
    (3): Linear(in_features=1024, out_features=1000, bias=True

Additionally, is the MobileNet_V3_Small_Weights version dependent? It gives me and ImportError.

4

The 1000 comes from the last layer in the classifier, as you can see there are 1000 out_features. The 2 layers you add will basically be appended right below that (3) Linear, as two children.

Yes, you should have v0.13 or later.

PS: In theory, you should replace the last Linear layer in the model_resnet with your two layers.

5

Thanks for the clarification Gian. Given the classifier has a sequence of 4 components, how does the indexing work to make the replacement work to insert two two layers? It’s one part I still having trouble understanding.

6

You can replace a layer like this:

model.classifier[3] = NewLayer(...)

Whilst you can remove it like this according to the implementation:

model.classifier.__delitem__(3)

Replacing won’t work in your case because you have two heads, what you can do is delete the last layers and then keep your code as it is. You just need to change from 1000 to 1024 input features.