NotImplementedError: Module [ModuleList] is missing the required "forward" function

1

Im trying to build network which extract features by backbone network (ViT) and classify by linear layer.
So i get error when trying to predict/forward-pass model:
raise NotImplementedError(f"Module [{type(self).name}] is missing the required "forward" function")
NotImplementedError: Module [ModuleList] is missing the required “forward” function

class FTNet(nn.Module):
    def __init__(self):
        super(FTNet, self).__init__()
        self.features = torch.nn.Sequential(*list(model_bb.children())[:-2])
        self.linear = torch.nn.Linear(320, 10)
        
    def forward(self, x):
        x = self.features(x)
        x = self.linear(x)
        return x
2

It seems you are using an nn.ModuleList in your model and are trying to call it directly which won’t work as it’s acting as a list but properly registers trainable parameters:

modules = nn.ModuleList([
    nn.Linear(10, 10),
    nn.ReLU(),
    nn.Linear(10, 10),
])
x = torch.randn(1, 10)
out = modules(x)
# NotImplementedError: Module [ModuleList] is missing the required "forward" function

You should iterate the modules instead:

out = x
for module in modules:
    out = module(out)

or use nn.Sequential:

model = nn.Sequential(
    nn.Linear(10, 10),
    nn.ReLU(),
    nn.Linear(10, 10),
)
x = torch.randn(1, 10)
out = model(x)

or via:

model = nn.Sequential(*modules)
out = model(x)
2 Likes
3

Hi. I tried, and it didnt work.

import torch
from src.models.backbone.load_models import init_zoo_model
model_bb = init_zoo_model(name_model='tiny_vit_5m_224', pretrained=True, device=torch.device('cpu'))
mod = list(model_bb.children())[:-2]
mod.append(torch.nn.Linear(320, 10))
model = torch.nn.Sequential(*mod)
x = torch.randn(1, 3, 224, 224)
res = model(x)
4

The main goal to remove 2 last layers from pretrained network:

  1. use it for finetune, so I can just replace last linear layer
  2. use nn as feature extractor as part of another neural architecture.

I can use this case:

model_bb = init_zoo_model(name_model='tiny_vit_5m_224', pretrained=True, device=torch.device('cpu'))
model_bb.norm_head = torch.nn.Identity()
model_bb.head = torch.nn.Linear(320, 10)
x = torch.randn(1, 3, 224, 224)
res = model_bb(x)

But i have no idea how to use it with proper nn class creation with forward call .

5

This solution is working

model_bb.norm_head = torch.nn.Identity()
model_bb.head = torch.nn.Identity()

# PyTorch models inherit from torch.nn.Module
class FTNet(torch.nn.Module):
    def __init__(self):
        super(FTNet, self).__init__()
        self.features = model_bb
        self.linear = torch.nn.Linear(320, 10)

    def forward(self, x):
        x = self.features(x)
        x = self.linear(x)
        return x

model = FTNet()
x = torch.randn(1, 3, 224, 224)
res = model(x)