AutoEncoder Removing intermediate layers causing size mismatch

Hello Experts,

I’m quite new to torch, so maybe it’s very simple. I’ve trained my model for encoding, which is:

class SAE(nn.Module):
    def __init__(self, ):
        super(SAE, self).__init__()
        self.fc1 = nn.Linear(n_features, 14)
        self.fc2 = nn.Linear(14, 10)
        self.fc3 = nn.Linear(10, 6)               
        self.fc4 = nn.Linear(6, 10)        
        self.fc5 = nn.Linear(10, 14)
        self.fc6 = nn.Linear(14, n_features)
        self.activation = nn.Sigmoid()
    def forward(self, x):
        x = self.activation(self.fc1(x))
        x = self.activation(self.fc2(x))
        x = self.activation(self.fc3(x))
        x = self.activation(self.fc4(x))
        x = self.activation(self.fc5(x))
        x = self.fc6(x)
        return x

then I tried to remove first three layers and create new model for decoding (used flattening from the different thread Removing Last Layer causing size mismatch

class Flatten(nn.Module):
    def __init__(self):
        super(Flatten, self).__init__()
        
    def forward(self, x):
        return x.view(x.size(0), -1)
new_model = nn.Sequential(*[Flatten(), *list(model.children())[3:]])

but still getting the size mismatch when trying to input intermediate layer output to the new model

def get_activation(name):
    def hook(model, input, output):
        activation[name] = output.detach()
    return hook
model.fc3.register_forward_hook(get_activation('fc3'))
encoded = model(input)
print(activation['fc3'])

How did you define input?
If you would like to feed the activation into your new model, you should use something like:

output = new_model(activation['fc3'])

Input is taken from intermediate layer of the model:

model.fc3.register_forward_hook(get_activation('fc3'))
encoded = model(input)

and then passed to the new model:

new_model = nn.Sequential(*[Flatten(), *list(model.children())[3:]])
decoded = new_model(encoded)

encoded is the model output, not the intermediate activation. The intermediate activation is stored in a dict called activation.

Thank you for the clarification… :slight_smile:
It works now with :
encoded = activation['fc3']