I have a model that stacks multiple BiLSTM layers over each other, where the output of one layer is given as input to the next. I want to perform a subsampling between 2 successive layers.
Currently, I am doing something like this:
output0, hidden_t = rnn0(input0, hidden) input1 = output0[::2] #output0 is of size (sequence_length, batch_size, hidden_dim) #input1 is of size (sequence_length/2, batch_size, hidden_dim) output1, hidden_t = rnn0(input1, hidden) input2 = output1[::2]
output1 etc are Variables, I am unsure if simply subsampling using
::2 is ok to do. If this is an incorrect approach, what would be the correct way of doing this kind of sampling? Is there a convolution layer I can use to obtain this result?