# How to use shared weights in different layers of a model

**URL:** <https://discuss.pytorch.org/t/how-to-use-shared-weights-in-different-layers-of-a-model/71263>\
**Category:** Uncategorized\
**Created:** [February 27, 2020, 9:06am UTC](https://discuss.pytorch.org/t/how-to-use-shared-weights-in-different-layers-of-a-model/71263 "2020-02-27T09:06:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dongyue\_Oliver](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dongyue_oliver/32/17625_2.png) [@Dongyue\_Oliver](https://discuss.pytorch.org/u/Dongyue_Oliver)\
**Post date:** [February 27, 2020, 9:06am UTC](https://discuss.pytorch.org/t/how-to-use-shared-weights-in-different-layers-of-a-model/71263/1 "2020-02-27T09:06:44Z")

</div>

I am trying to share the weights in different layers in one model. Please take a look at this example code:

this code tries to share weights in fc1 and some parts of fc2

```
import torch
import torch.nn as nn
import torch.optim as optim

class testModule(nn.Module):

    def __init__ (self):
        super(testModule, self). __init__ ()
        self.fc1 = nn.Linear(5, 10, bias=True)
        self.fc2 = nn.Linear(10, 10, bias=False)

        self.shared_weights = nn.Parameter(torch.randn(10, 5), requires_grad=True)

    def forward(self, x):
        x = self.fc1(x)
        x = self.fc2(x)
        return x

    def share_weight(self):
        index = [1, 3, 5, 7, 9]
        self.fc1.weight.data = self.shared_weights
        self.fc2.weight.data[:, index] = self.shared_weights

```

After calling the .share\_weight() method and training, the weight in fc1.weight and fc2.weight[:, index] become different.

Why would this happen and what is the behavior behind assigning weight.data as another tensor. How could I reach the goal of sharing fc1.weight and fc2.weight[:, index] in training.

---

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [February 27, 2020, 5:02pm UTC](https://discuss.pytorch.org/t/how-to-use-shared-weights-in-different-layers-of-a-model/71263/2 "2020-02-27T17:02:09Z")

</div>

Hi,

`.data` is in the process of being removed and should not be used. As you have experienced, it only does very confusing things 😃

You will need to have only `nn.Parameter`s to be the true parameters and you will have to recompute other things at each forward:

```python
import torch
import torch.nn as nn
import torch.optim as optim

class testModule(nn.Module):

    def __init__ (self):
        super(testModule, self). __init__ ()
        self.fc1 = nn.Linear(5, 10, bias=True)
        self.fc2 = nn.Linear(10, 10, bias=False)

        # Remove the weights as we override them in the forward
        # so that they don't show up when calling .parameters()
        del self.fc1.weight
        del self.fc2.weight

        self.fc2_base_weights = nn.Parameter(torch.randn(10, 10))
        self.shared_weights = nn.Parameter(torch.randn(10, 5))

    def forward(self, x):
        # Update the weights
        index = [1, 3, 5, 7, 9]
        self.fc1.weight = self.shared_weights
        self.fc2.weight = self.fc2_base_weights.clone()
        self.fc2.weight[:, index] = self.shared_weights

        x = self.fc1(x)
        x = self.fc2(x)
        return x

    def _weight(self):

```

---

<div class="post-metadata">

**Author:** ![Dongyue\_Oliver](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dongyue_oliver/32/17625_2.png) [@Dongyue\_Oliver](https://discuss.pytorch.org/u/Dongyue_Oliver)\
**Post date:** [February 28, 2020, 2:41am UTC](https://discuss.pytorch.org/t/how-to-use-shared-weights-in-different-layers-of-a-model/71263/3 "2020-02-28T02:41:31Z")

</div>

Thank you! This way works!