LSTM layer with masking like in Lasagne/Theano

himat · February 6, 2018, 10:19pm

I’m converting some Theano/Lasagne code to PyTorch, but I’m unsure if an LSTM with masking is possible in PyTorch.
The line in the Theano code is
l_box_lstm = lasagne.layers.LSTMLayer(l_box, num_units=d_word, mask_input=l_bbmask, only_return_final=True)
d_word = 256

Here, l_box is of shape mb_size x seq_len x features_size = 192 x 3 x 256, and l_bbmask is of shape mb_size x seq_len = 192 x 3

I was looking at how the mask_input is used in the LSTM layer, and it seems like it’s not so simple.

github.com

Lasagne/Lasagne/blob/master/lasagne/layers/recurrent.py#L931


    # of input_shapes, whether or not a mask input is being used.
    input_shape = input_shapes[0]
    # When only_return_final is true, the second (sequence step) dimension
    # will be flattened
    if self.only_return_final:
        return input_shape[0], self.num_units
    # Otherwise, the shape will be (n_batch, n_steps, num_units)
    else:
        return input_shape[0], input_shape[1], self.num_units


def get_output_for(self, inputs, **kwargs):
    """
    Compute this layer's output function given a symbolic input variable


    Parameters
    ----------
    inputs : list of theano.TensorType
        `inputs[0]` should always be the symbolic input variable.  When
        this layer has a mask input (i.e. was instantiated with
        `mask_input != None`, indicating that the lengths of sequences in
        each batch vary), `inputs` should have length 2, where `inputs[1]`

Does anyone know if I can just expand the shape of l_bbmask to also be 192 x 3 x 256 and elementwise multiply it with l_box to get the same result, or no? It seems like the lasagne code uses it at every step, but I’m not really sure since I don’t really understand the lasagne code that well.