Define the number of in_feature in nn.Linear dynamically

vision
1

Hi!
I have a situation where I need to flatten and concatenate the feature maps of each view from the last conv layer followed by a Linear layer.

 def forward(self, views):  # views = (B x n_views x C x H x W)
        views = views.transpose(0, 1)  # (n_views x B x C x H x W)
        aggregated_view = []
        for x in views:  # (B x C x H x W)
            x = self.layer1(x)
            x = self.layer2(x)
            x = self.layer3(x)
            x = self.layer4(x)
            x = self.avgpool(x)
            x = x.view(x.size(0), -1) # Flatten
            aggregated_view.append(x)
        x = torch.cat(aggregated_view, dim=1)
        x = self.fc(x)
        return x

My problem, is how to defined self.fc since the number of views might vary. If it was fixed, It could be defined as

def __init__(self, n_views, n_classe):
      .....
      .....
    self.fc = nn.Linear(512*n_views, n_classe)

I know pytorch builds the graph dynamically, Can i define the number of in_features for a linear layer in the forward function and still have the param of this layer updated?

2

Is the number of views known before the forward pass or is it changing for each batch?
In the former case you could just pass the number of views to your model’s __init__ method and define the linear layer there as usual.
In the latter case, you could try to use an adaptive pooling layer with a specific output size, so that your linear layer will get the same number of features for differently shaped inputs.
As you said, you could also use the functional API and define your linear weights as nn.Parameters in your model. However, I’m not sure how you would like to change these during training.
In case the next input would be larger, would you like to initialize the new values randomly while the other part of your weight matrix has already been trained?

3

Thank you. It works, I used an adaptive pooling layer before the linear layer.