Forgetting some information about some specific input elements in LSTM hidden state

I have an LSTM network and I want to forget some information from prev time steps. I mean I don’t want to reset the whole hidden state, but just want to do not consider the history of some elements of input vector. Does anyone have any idea?

Hidden state entangles input vector elements, so this is impossible I think. You’d have to somehow split your vector before rnn layer, and conditionally combine it back to produce rnn input (residual networks, multi-layer rnns, something like that…)

1 Like