# Confused about forward function of custom layer

**URL:** <https://discuss.pytorch.org/t/confused-about-forward-function-of-custom-layer/20698>\
**Category:** autograd\
**Created:** [July 4, 2018, 12:42pm UTC](https://discuss.pytorch.org/t/confused-about-forward-function-of-custom-layer/20698 "2018-07-04T12:42:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Naruto-Sasuke](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/ed8c4c/32.png) [@Naruto-Sasuke](https://discuss.pytorch.org/u/Naruto-Sasuke)\
**Post date:** [July 4, 2018, 12:42pm UTC](https://discuss.pytorch.org/t/confused-about-forward-function-of-custom-layer/20698/1 "2018-07-04T12:42:30Z")

</div>

This is code snippet from PyTorch tutorial.

```auto
# Residual block
class ResidualBlock(nn.Module):
    def __init__ (self, in_channels, out_channels, stride=1, downsample=None):
        super(ResidualBlock, self). __init__ ()
        self.conv1 = conv3x3(in_channels, out_channels, stride)
        self.bn1 = nn.BatchNorm2d(out_channels)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = conv3x3(out_channels, out_channels)
        self.bn2 = nn.BatchNorm2d(out_channels)
        self.downsample = downsample
        
    def forward(self, x):
        residual = x
        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)
        out = self.conv2(out)
        out = self.bn2(out)
        if self.downsample:
            residual = self.downsample(x)
        out += residual
        out = self.relu(out)
        return out

```

`out` overwrites the previous `out`. Then how can the gradients successfully back-propagate?

---

<div class="post-metadata">

**Author:** ![Shani\_Gamrian](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shani_gamrian/32/1299_2.png) [@Shani\_Gamrian](https://discuss.pytorch.org/u/Shani_Gamrian)\
**Post date:** [July 4, 2018, 1:45pm UTC](https://discuss.pytorch.org/t/confused-about-forward-function-of-custom-layer/20698/2 "2018-07-04T13:45:43Z")

</div>

`out` contains a list of layers it passed through the feed-forward process, every time you do `out = layer(out)`, the `layer` is added to the list. Once you back-propagate it goes through the list from the last to the first on the list.