# When to implement backward()?

**URL:** <https://discuss.pytorch.org/t/when-to-implement-backward/98067>\
**Category:** Uncategorized\
**Created:** [October 2, 2020, 10:35pm UTC](https://discuss.pytorch.org/t/when-to-implement-backward/98067 "2020-10-02T22:35:49Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![amirhf](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/amirhf/32/5634_2.png) [@amirhf](https://discuss.pytorch.org/u/amirhf)\
**Post date:** [October 2, 2020, 10:35pm UTC](https://discuss.pytorch.org/t/when-to-implement-backward/98067/1 "2020-10-02T22:35:49Z")

</div>

I am reading through the [‘Extending PyTorch’](https://pytorch.org/docs/stable/notes/extending.html) that implements a new `nn.Module` and uses the function that has both `forward()` and `backward()` implemented. My question is if I want to define a new module that just adds a random weight to the input tensor and I define it like this:

```auto
class CustomLayer(nn.Module):
    def __init__ (self, input_dim):
        super(CustomLayer, self). __init__ ()
        self.input_dim = input_dim
        self.weight = nn.Parameter(torch.Tensor(input_dim))
        self.weight.data.uniform_(-0.05, 0.05)

    def forward(self, input):
        return torch.add(input, self.weight)

```

Then do I need to implement a `backward()` as well? Because in the forward method, I am not using a `Function`, but just an already implemented operation in PyTorch.  
Does having an `nn.Parameter()` in my contructor requires me to write a `backward()` function? Do they get updated without havin a `backward()`?  
To my understanding if only you implement a custom function and you want to somehow manipulate the backward operations, then you need to write a `backward()` function. So for example, the module in the tutorial can be simply written as:

```auto
class Linear(nn.Module):
    def __init__ (self, input_features, output_features, bias=True):
        super(Linear, self). __init__ ()
        self.input_features = input_features
        self.output_features = output_features
        self.weight = nn.Parameter(torch.Tensor(output_features, input_features))
        if bias:
            self.bias = nn.Parameter(torch.Tensor(output_features))
        else:
            self.register_parameter('bias', None)

        self.weight.data.uniform_(-0.1, 0.1)
        if self.bias is not None:
            self.bias.data.uniform_(-0.1, 0.1)

    def forward(self, input):
        return torch.add(torch.matmul(input, self.weight), self.bias)

```

without a `backward()` method!

---

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [October 3, 2020, 4:36pm UTC](https://discuss.pytorch.org/t/when-to-implement-backward/98067/2 "2020-10-03T16:36:16Z")

</div>

Hi,

Yes you only need to write a backward function if:

- You want to compute something that is not the “true” gradient that is computed by the autograd
- Your forward function is not handled by the autograd (because you use custom function or third party library)

And the example in the tutorial is indeed artificial but it is just an example on how to do that for a simple function.

---

<div class="post-metadata">

**Author:** ![amirhf](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/amirhf/32/5634_2.png) [@amirhf](https://discuss.pytorch.org/u/amirhf)\
**Post date:** [October 4, 2020, 4:32am UTC](https://discuss.pytorch.org/t/when-to-implement-backward/98067/3 "2020-10-04T04:32:18Z")

</div>

That makes sense. Thank you for the clarification!
