Deal with variable length sequence for 1D convolution & batch normalization

Hi.

Is there any example or related document which describes how to deal with variable length sequences in minibatch for 1D convolution?

Here is my detail situation.

I have two sequences with size : (#Channel, #Length).
Each pair of sequence have same length, but it differs within dataset.
For example, let’s say (X1,Y1), (X2, Y2) is paired data with size
X1.size() --> [5, 10], Y1.size() --> [3, 10]
X2.size() --> [5, 20], Y2.size() --> [3, 20]
My goal is learning the model f such that Y = f(X)

I am considering f as 1D convolution with batch normalization like,
h1 = bn(relu(conv(input)))
h2 = bn(relu(conv(h1)))

However, I am confused how to deal with multiple sequences in minibatch. If we zero-pad sequence when making minibatch, there seems no example to exclude zero-pad data involved in computation.

if you zero-pad the data, and then subsequently narrow the output to avoid the padding-contributed regions, this should be sufficient.

2 Likes