# Model parameters as sum of variable and constant weight

Hello everyone,
I am trying to train a model that uses parameters that are a sum of two sets weights and I want to to train only one set of weights. That is, for every weight w = w_1 + w_2, where w_2 is a constant and w_1 is to be optimized using an optimizer. Thus, in the forward step w is used to compute the loss, but in the backward step only w_1 is adjusted. Is this possible?

It is possible.

Just make `w1.requires_grad = True ~~ w1.requires_grad_(True)` and `w2.requires_grad_(False)`.

Look at the example below.

``````import torch
class Model(torch.nn.Module) :
def __init__(self) :
super().__init__()
self.w1 = torch.nn.parameter.Parameter(data = torch.tensor(1.), requires_grad = True)
self.w2 = torch.nn.parameter.Parameter(data = torch.tensor(2.), requires_grad = False)

def forward(self, x) :
w = self.w1 + self.w2
return w * x

def mae(y, y_pred) :
"""mean absolute error"""
return (y - y_pred).abs().mean()

# data
x = torch.tensor([1., 3, 3])
y = 2 * x
# model
model = Model()

In this case, `optimizer.step()` will update the parameters. For example : `w1 = w1 - learning_rate*w1.grad != w1` and `w2 = w2 - learning_rate*w2.grad = w2 - 0 = w2` (In fact, optimizing it won’t even do that for w2, since w2 will be seen in the computation graph as a constant)