# How to fix the weights and biases except two?

**URL:** <https://discuss.pytorch.org/t/how-to-fix-the-weights-and-biases-except-two/208515>\
**Category:** Uncategorized\
**Created:** [August 23, 2024, 7:59am UTC](https://discuss.pytorch.org/t/how-to-fix-the-weights-and-biases-except-two/208515 "2024-08-23T07:59:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![cnn](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cnn/32/72148_2.png) [@cnn](https://discuss.pytorch.org/u/cnn)\
**Post date:** [August 23, 2024, 7:59am UTC](https://discuss.pytorch.org/t/how-to-fix-the-weights-and-biases-except-two/208515/1 "2024-08-23T07:59:09Z")

</div>

After training a model, how can I save the parameters of the network (e.g., weights and biases) and apply them to a new equal network BUT where I want to change only the first two weights?

I would like to see how the loss changes when only the first two weights varies (providing a meshgrid space) and all the others are fixed.

1. How to fix the weights and biases except two?
2. How to provide the domain in which the two free weights can span?

```auto
class Net(nn.Module):

    def __init__ (self, layer_sizes, activation=torch.tanh):
        super(Net, self). __init__ ()
        self.layer_sizes = layer_sizes
        self.layer_list = [x]
        self.activation = activation

        self.l1 = nn.Linear(layer_sizes[0], layer_sizes[1])
        self.l2 = nn.Linear(layer_sizes[1], layer_sizes[2])
        self.l3 = nn.Linear(layer_sizes[2], layer_sizes[3])
        self.l4 = nn.Linear(layer_sizes[3], layer_sizes[4])

    def forward(self, x):
        x = self.l1(x)
        #print(x.dtype)
        x = self.activation(x)
        x = self.l2(x)
        x = self.activation(x)
        x = self.l3(x)
        x = self.activation(x)
        x = self.l4(x)
        return x

device = 'cpu'
layer_sizes = [1,32,32,32,1]
activation = torch.tanh

net_u = Net(layer_sizes, activation).to(device)

... # training

# saving 
info2exp = []
for layer in nets[0].children():
    if isinstance(layer, nn.Linear):
        info2exp.append(layer.state_dict())

torch.save([data, epochs, info2exp], r'./dataStorage/test_v1.pt')

```

How to load them to a new network? Like this?

```auto
net_new = Net(layer_sizes, activation).to(device)
net_new.layers.load_state_dict(net_u.layers.state_dict())

print((net_new.layers[0].weight == net_u.layers[0].weight).all())

```

1. How to fix the weights and biases except two?
2. How to provide the domain in which the two free weights can span?  
Thanks!

---

<div class="post-metadata">

**Author:** ![JuanFMontesinos](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/32/76115_2.png) [@JuanFMontesinos](https://discuss.pytorch.org/u/JuanFMontesinos)\
**Post date:** [August 23, 2024, 8:38am UTC](https://discuss.pytorch.org/t/how-to-fix-the-weights-and-biases-except-two/208515/2 "2024-08-23T08:38:39Z")

</div>

Hmmm  
Esentially what you save is a dicitonary.  
The simplest way is changing the values in the state\_dict before the new network loads them.  
Something like

```python
weights = net.state_dict()
weights['layer_name1'] = ... 
weights['layer_nameN'] = ...
new_net.load_state_dict(weights)

```

To freeze certain layers: [How the pytorch freeze network in some layers, only the rest of the training?](https://discuss.pytorch.org/t/how-the-pytorch-freeze-network-in-some-layers-only-the-rest-of-the-training/7088)

---

<div class="post-metadata">

**Author:** ![J\_Johnson](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/j_johnson/32/55494_2.png) [@J\_Johnson](https://discuss.pytorch.org/u/J_Johnson)\
**Post date:** [August 24, 2024, 5:17am UTC](https://discuss.pytorch.org/t/how-to-fix-the-weights-and-biases-except-two/208515/3 "2024-08-24T05:17:36Z")

</div>

A1. If you want to stop training on any layer of a model, just set the `requires_grad` variable to `False`. For example:

```auto
model.l1.weight.requires_grad = False
model.l1.bias.requires_grad = False

```

A2. Not clear on what you’re asking.