Masking layer for RNN

Mostafa_Alishahi · December 31, 2019, 2:24pm

Is there any function similar to keras.layers.Masking which masks the rows of timeseries data which is filled with an specific value (e.g. padded with zero)?

G.M · December 31, 2019, 2:56pm

By slicing perhaps.

t=tc.tensor(B,L,D)
t[ : , a:b , : ]=0

Mostafa_Alishahi · December 31, 2019, 3:10pm

Thanks for your answer. in this way, I need to have the start and end index of the non padded values, isn’t there any other more straightforward way like Masking layer in keras?

one thing more, by slicing we feed the matrix with different shapes into RNN, is this acceptable in pytorch? (e.g. data[0].size() could be (timesteps=100, features=20) and data[10].size() could be (timesteps=34, features=20))

G.M · December 31, 2019, 3:15pm

Idk how r you going to set values to zero if u don’t know where the values are. You can check torch.masked_select, but i’m quite sure u need to know what to mask. Just curious, are you doing ASR?
u have to pad them into a bigger tensor, u can’t even feed it into rnn if u don’t pad

Mostafa_Alishahi · December 31, 2019, 3:50pm

Ok, Thanks for the clarifications . No I am not doing ASR.

hadaev8 · December 31, 2019, 5:14pm

Use this

gist.github.com

https://gist.github.com/Tushar-N/dfca335e370a2bc3bc79876e6270099e

pad_packed_demo.py

import torch
import torch.nn as nn
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence

seqs = ['gigantic_string','tiny_str','medium_str']

# make <pad> idx 0
vocab = ['<pad>'] + sorted(set(''.join(seqs)))

# make model

This file has been truncated. show original

Sanbot-kk · June 10, 2021, 8:49am

Hello,

Were you able to find a solution for this?