Sequence to Sequence with attention on continuous valued input and output

nlp
1

Hi all,
Thus far, all the examples of seq2seq with attention (like language translation) I have seen are for categorical inputs. I was wondering if we can use seq2seq with attention on continuous input/output. A trivial example would be to Denoise a sine wave.

1 Like
2

yes you can use seq2seq for continuous inputs. Instead of having the encoder / decoder be Embedding / Linear/Softmax, you will have simple Linear layers there (regression loss)

3

It’s the attention mechanism for se2seq for continuous inputs that I’m
unsure about.

4

it’s the same concept, i dont think it would differ on continuous vs categorical