# Autoencoders in Pytorch

**URL:** <https://discuss.pytorch.org/t/autoencoders-in-pytorch/844>\
**Category:** Uncategorized\
**Created:** [March 2, 2017, 10:40pm UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844 "2017-03-02T22:40:58Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmadeka1](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dmadeka1/32/18387_2.png) [@dmadeka1](https://discuss.pytorch.org/u/dmadeka1)\
**Post date:** [March 2, 2017, 10:40pm UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/1 "2017-03-02T22:40:58Z")

</div>

Would Pytorch support something like this? How does one go about implementing a simple Autoencoder?

```python
class Encoder(nn.Module):
    def __init__ (self):
        super(Encoder, self). __init__ ()
        self.fc1 = nn.Linear(784, 32)

    def forward(self, x):
        return F.sigmoid(self.fc1(x))

class Decoder(nn.Module):
    def __init__ (self):
        super(Decoder, self). __init__ ()
        self.fc1 = nn.Linear(32, 784)
        
    def forward(self, x):
        return F.sigmoid(self.fc1(x))

class AutoEncoder(nn.Module):
    def __init__ (self):
        super(AutoEncoder, self). __init__ ()
        self.fc1 = Encoder()
        self.fc2 = Decoder()

    def forward(self, x):
        return self.fc2(self.fc1(x))

model = AutoEncoder()
optimizer = optim.Adam(model.parameters(), lr=0.5)
for epoch in range(1, 201):
    train(epoch)
    test(epoch, validation)

```

---

<div class="post-metadata">

**Author:** ![dmadeka1](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dmadeka1/32/18387_2.png) [@dmadeka1](https://discuss.pytorch.org/u/dmadeka1)\
**Post date:** [March 2, 2017, 11:08pm UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/2 "2017-03-02T23:08:13Z")

</div>

Just looking for the simplest possible implementation of an AutoEncoder here

---

<div class="post-metadata">

**Author:** ![apaszke](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/apaszke/32/21_2.png) [@apaszke](https://discuss.pytorch.org/u/apaszke)\
**Post date:** [March 2, 2017, 11:38pm UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/3 "2017-03-02T23:38:51Z")

</div>

Yes, that should work.

---

<div class="post-metadata">

**Author:** ![alexis-jacq](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alexis-jacq/32/180_2.png) [@alexis-jacq](https://discuss.pytorch.org/u/alexis-jacq)\
**Post date:** [March 3, 2017, 12:08am UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/4 "2017-03-03T00:08:33Z")

</div>

If you really want to do the simplest, I would suggest:

```
class Autoencoder(nn.Module):
    def __init__ (self, ):
        super(Autoencoder, self). __init__ ()
        self.fc1 = nn.Linear(784, 32)
        self.fc2 = nn.Linear(32, 784)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        x = self.sigmoid(self.fc1(x))
        x = self.sigmoid(self.fc2(x))
        return x
```

---

<div class="post-metadata">

**Author:** ![dmadeka1](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dmadeka1/32/18387_2.png) [@dmadeka1](https://discuss.pytorch.org/u/dmadeka1)\
**Post date:** [March 3, 2017, 1:02am UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/5 "2017-03-03T01:02:55Z")

</div>

@alexis-jacq I need to access the intermediate data… Can I do that in your implementation?

@apaszke I thought it would work too, but it says

```python
matrices expected, got 4D, 2D tensors at /data/users/soumith/miniconda2/conda-bld/pytorch-0.1.7_1485444530918/work/torch/lib/TH/generic/THTensorMath.c:857

```

---

<div class="post-metadata">

**Author:** ![alexis-jacq](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alexis-jacq/32/180_2.png) [@alexis-jacq](https://discuss.pytorch.org/u/alexis-jacq)\
**Post date:** [March 3, 2017, 10:36am UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/6 "2017-03-03T10:36:16Z")

</div>

> [@dmadeka1](#):
>
> @alexis-jacq I need to access the intermediate data… Can I do that in your implementation?

In that case your approach seems simpler. You can even do:

```
encoder = nn.Sequential(nn.Linear(782,32), nn.Sigmoid())
decoder = nn.Sequential(nn.Linear(32,732), nn.Sigmoid())
autoencoder = nn.Sequential(encoder, decoder)

```

---

<div class="post-metadata">

**Author:** ![Wong\_Fungtion](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/w/4bbf92/32.png) [@Wong\_Fungtion](https://discuss.pytorch.org/u/Wong_Fungtion)\
**Post date:** [August 30, 2017, 8:18am UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/8 "2017-08-30T08:18:54Z")

</div>

@alexis-jacq I want a auto encoder with tied weights, i.e. weight of encoder equal with decoder. How to implement it?

---

<div class="post-metadata">

**Author:** ![alexis-jacq](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alexis-jacq/32/180_2.png) [@alexis-jacq](https://discuss.pytorch.org/u/alexis-jacq)\
**Post date:** [September 11, 2017, 4:35pm UTC](https://discuss.pytorch.org/t/autoencoders-in-pytorch/844/9 "2017-09-11T16:35:02Z")

</div>

So you want a kind of balanced autoencoder, where Encoder = Transpose(Decoder)? In that case, I would do something like this:

```python
class BalancedAE(nn.Module):
    def __init__ (self, ):
        super(BalancedAE, self). __init__ ()
        self.encoder = nn.Parameter(torch.rand(size_input, size_output))

    def forward(self, x):
        x = torch.sigmoid(torch.mm(self.encoder, x))
        x = torch.sigmoid(torch.mm(x, torch.transpose(self.encoder, 0, 1))
        return x

```