Variable Input Size

I have an input that its size is varied e.g. (102 x 128) or (102 x 1100).

I create a network that is theoretically valid on this different size of input as follows
(In my model class, I define forward method as follows)

def forward(self, inputs):
        x = self.Batchnorm1(inputs)
        x, _ = self.LSTM(x)
        x, _  = torch.max(x, 1)
        x = self.linear(x)
        x = torch.sigmoid(x)

That is by using global max pooling, the dimension will match.

Besides, I also make some customization on the dataloader for training different size of inputs in the batch according to https://jdhao.github.io/2017/10/23/pytorch-load-data-and-make-batch/

However, the problem still persists. That is the output from new customized collate_fn which is as fllows

def my_collate(batch):
    data = [item[0] for item in batch]  # just form a list of tensor

    target = [item[1] for item in batch]
    target = torch.LongTensor(target)
    return [data, target]

will still output the type of input data that I cannot feed into batch normalization (or other types of layers) in my forward function since it expects 3-d Tensor (Batch Size x dim1x dim2)
How should I make any changes to my model to be able to receive the input.

ps. padding is not an option here

Hi @tsut,

I think that if you cannot pad your data, you won’t have other solutions that doing:

outputs = []
for datum in data:
    outputs.append(model(datum)) # you might need to add `.unsqueeze(0)` to datum
outputs = torch.stack(outputs)
1 Like

Thanks for you reply.

By doing like that (feeding in 1-1 vs feeding entire batch at once), will I still have the same effect of batch normalization layer ?

Oh… no it’s not. I’m sorry, I missed your point about Batchnorm here.
I don’t thinks it is possible to use the default batchnorm here. You would need to have your input data projected into same size tensors (some interpolation method?).