# How To Implement Custom Layer And Update weight In Pytorch

**URL:** <https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076>\
**Category:** Uncategorized\
**Created:** [September 29, 2017, 2:35pm UTC](https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076 "2017-09-29T14:35:31Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![herleeyandi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/herleeyandi/32/948_2.png) [@herleeyandi](https://discuss.pytorch.org/u/herleeyandi)\
**Post date:** [September 29, 2017, 2:35pm UTC](https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076/1 "2017-09-29T14:35:31Z")

</div>

Hello,  
I am trying to create some layers. Let we said convolutional layer but will have some logic operation inside of it. Basically I can do that using tensor operation and create it as function inside the class. However I am so confuse how can I update the weight of my custom convolution filter?  
As far as I know most of people implement the new layer and they will still using `torch.nn.functional` so how can I do this?

I have read some threads like [here](https://discuss.pytorch.org/t/implementing-a-custom-convolution-layer/5659) , [here](https://discuss.pytorch.org/t/implementing-custom-layer-for-cnn/3702) , also [here](http://pytorch.org/docs/master/notes/extending.html) .

Or do we just use `nn.Parameter` is enough like in [here](https://discuss.pytorch.org/t/how-to-define-a-new-layer-with-autograd/351/2) .

I am still confuse where to start, suggestion to start is welcome.

---

<div class="post-metadata">

**Author:** ![pvskand](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3be4f8/32.png) [@pvskand](https://discuss.pytorch.org/u/pvskand)\
**Post date:** [September 29, 2017, 2:56pm UTC](https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076/2 "2017-09-29T14:56:25Z")

</div>

Implementing the custom layer : [http://pytorch.org/tutorials/beginner/pytorch\_with\_examples.html#pytorch-custom-nn-modules](http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-custom-nn-modules)  
This is a very good reference and in fact it was present in one of the links you mentioned.  
You basically need to write the implementation of your custom layer in the `forward()` function.  
The weight update would be taken care by the autograd.

I hope this helps.

---

<div class="post-metadata">

**Author:** ![nabsabs](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/c77e96/32.png) [@nabsabs](https://discuss.pytorch.org/u/nabsabs)\
**Post date:** [March 6, 2019, 4:29am UTC](https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076/3 "2019-03-06T04:29:51Z")

</div>

Hi Skand,  
Do you know how we can go about creating a custom layer that does not need a backward method? The gradients should just flow through it when back propagating. Any help appreciated!

---

<div class="post-metadata">

**Author:** ![pvskand](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3be4f8/32.png) [@pvskand](https://discuss.pytorch.org/u/pvskand)\
**Post date:** [March 6, 2019, 4:47am UTC](https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076/4 "2019-03-06T04:47:46Z")

</div>

Hi @nabsabs,  
You can simply freeze the `layer` weights if you don’t want any change in them. The back propogation would not change any value of that layer’s weights.

---

<div class="post-metadata">

**Author:** ![nabsabs](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/c77e96/32.png) [@nabsabs](https://discuss.pytorch.org/u/nabsabs)\
**Post date:** [March 7, 2019, 1:44am UTC](https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076/5 "2019-03-07T01:44:16Z")

</div>

Hi @pvskand thanks for the quick reply! Can you elaborate, maybe with an example, on how to freeze the layers? If I don’t write a backward method, does that count as not changing the value of the layer’s weights?

---

<div class="post-metadata">

**Author:** ![pvskand](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/3be4f8/32.png) [@pvskand](https://discuss.pytorch.org/u/pvskand)\
**Post date:** [March 7, 2019, 6:17am UTC](https://discuss.pytorch.org/t/how-to-implement-custom-layer-and-update-weight-in-pytorch/8076/6 "2019-03-07T06:17:40Z")

</div>

You can have a look at this [link](https://discuss.pytorch.org/t/how-to-freeze-a-specific-layer-in-pytorch/14868) for freezing of weights.  
No, not writing a backward method does not necessarily mean that the weights would not change. If the `forward` method of your layer has differentiable operations then, autograd takes care of backward by default.