# How to customize the "backward" function to not calculate gradients and not update a selected sub-tensor of a parameter tensor of a layer

**URL:** <https://discuss.pytorch.org/t/how-to-customize-the-backward-function-to-not-calculate-gradients-and-not-update-a-selected-sub-tensor-of-a-parameter-tensor-of-a-layer/169103>\
**Category:** autograd\
**Created:** [December 27, 2022, 8:46pm UTC](https://discuss.pytorch.org/t/how-to-customize-the-backward-function-to-not-calculate-gradients-and-not-update-a-selected-sub-tensor-of-a-parameter-tensor-of-a-layer/169103 "2022-12-27T20:46:01Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 27, 2022, 10:19pm UTC](https://discuss.pytorch.org/t/how-to-customize-the-backward-function-to-not-calculate-gradients-and-not-update-a-selected-sub-tensor-of-a-parameter-tensor-of-a-layer/169103/2 "2022-12-27T22:19:35Z")

</div>

> [@eric4](#):
>
> I don’t want to replace the gradients after being calculated by 0 because my goal is to avoid computing them for optimization reasons,

In that case you could create a “fixed” and a “trainable” tensor in a custom linear layer and concatenate them in each `forward` pass. This would make sure that only the trainable part gets valid gradient and the parameter updates.  
[This post](https://discuss.pytorch.org/t/custumize-the-backpropagation-phase-of-a-neural-network-to-ignore-the-update-of-some-parameters/167557/4) gives you an example of such a layer and replaces it via `torch.fx` in another model.

---

_[View the full topic](https://discuss.pytorch.org/t/how-to-customize-the-backward-function-to-not-calculate-gradients-and-not-update-a-selected-sub-tensor-of-a-parameter-tensor-of-a-layer/169103)._