# TCN autoencoder

**URL:** <https://discuss.pytorch.org/t/tcn-autoencoder/140635>\
**Category:** Uncategorized\
**Created:** [January 1, 2022, 7:14pm UTC](https://discuss.pytorch.org/t/tcn-autoencoder/140635 "2022-01-01T19:14:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![fatcat](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/94ad74/32.png) [@fatcat](https://discuss.pytorch.org/u/fatcat)\
**Post date:** [January 1, 2022, 7:14pm UTC](https://discuss.pytorch.org/t/tcn-autoencoder/140635/1 "2022-01-01T19:14:52Z")

</div>

Hello everyone,

I am going to use 1D convolutions to learn representations of time series data. I plan to use an Encoder-Decoder architecture. The input shape is like: [batch\_size, num\_features, num\_timesteps]; the outputs of the encoder should be like: [batch\_size, length]; i.e., I wish to get a fixed-length representation for each sequence. It seems that the length of the outputs depends on the original length of the sequence of the batch. Is there a method that I can define the length of the outputs of the encoder regardless of the length(num\_timesteps) of the original sequence?

---

<div class="post-metadata">

**Author:** ![googlebot](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/ba8739/32.png) [@googlebot](https://discuss.pytorch.org/u/googlebot)\
**Post date:** [January 2, 2022, 12:08pm UTC](https://discuss.pytorch.org/t/tcn-autoencoder/140635/2 "2022-01-02T12:08:06Z")

</div>

using mean (a la average pooling) is the usual reduction method for unknown lengths. but by itself it is too simplistic, i.e. it is a linear interpolation invariant to reordering. one possibility is to apply an attention layer first.

---

<div class="post-metadata">

**Author:** ![fatcat](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/94ad74/32.png) [@fatcat](https://discuss.pytorch.org/u/fatcat)\
**Post date:** [January 2, 2022, 2:00pm UTC](https://discuss.pytorch.org/t/tcn-autoencoder/140635/3 "2022-01-02T14:00:58Z")

</div>

Thanks for your reply. Do you mean that I should use an attention over all the time steps to get an attention vector whose shape is [batch\_size, 1, num\_timesteps] and then multiply this attention vector with another vector whose shape is [batch\_size, num\_channels, num\_timesteps] and then compute the mean along the 2nd dimension (num\_timesteps) to get an output vector whose shape is [batch\_size, num\_channels] ?

---

<div class="post-metadata">

**Author:** ![fatcat](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/f/94ad74/32.png) [@fatcat](https://discuss.pytorch.org/u/fatcat)\
**Post date:** [January 2, 2022, 2:50pm UTC](https://discuss.pytorch.org/t/tcn-autoencoder/140635/4 "2022-01-02T14:50:42Z")

</div>

And I am also a little confused about how can I transform the latent variables whose shape is [batch\_size, length] to time series in the decoder part.

---

<div class="post-metadata">

**Author:** ![googlebot](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/ba8739/32.png) [@googlebot](https://discuss.pytorch.org/u/googlebot)\
**Post date:** [January 3, 2022, 4:43am UTC](https://discuss.pytorch.org/t/tcn-autoencoder/140635/5 "2022-01-03T04:43:50Z")

</div>

Well, frankly, implementation details vary, some being pretty complex. I’d suggest to start from some existing design for a similar problem.

Your attention description sounds right, but it is pretty generic… For encoder, it can be a self-attention layer, to support unequal importance of time points. For some tasks, you’d want to encode positions too (and/or time-based features), for same reason as in NLP - to make element order matter.

As for decoders, I’m not too familiar with RNN-less context unrolling for convolutional decoders. I believe another attention module is one way to do it. Again, it may be better to check what they did in relevant research papers.