# Single weight-sharing across a network

**URL:** <https://discuss.pytorch.org/t/single-weight-sharing-across-a-network/67798>\
**Category:** Uncategorized\
**Created:** [January 27, 2020, 4:41pm UTC](https://discuss.pytorch.org/t/single-weight-sharing-across-a-network/67798 "2020-01-27T16:41:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chris\_Culley](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chris_culley/32/15130_2.png) [@Chris\_Culley](https://discuss.pytorch.org/u/Chris_Culley)\
**Post date:** [January 27, 2020, 4:41pm UTC](https://discuss.pytorch.org/t/single-weight-sharing-across-a-network/67798/1 "2020-01-27T16:41:39Z")

</div>

I’d like to force a single weight to be the same in two different linear layers and have them optimise jointly.

I’ve tried the following, but now I can’t optimise since the ‘hidden\_layer’ is no longer a leaf node.

```auto
class Net(nn.Module):
  def __init__ (self, dim_in, dim_out, dim_hidden):
    super(Net, self). __init__ ()
    self.input_layer = nn.Linear(dim_in, dim_hidden)
    self.hidden_layer = nn.Linear(dim_hidden, dim_hidden)
    self.hidden_layer.weight[0,0] = self.input_layer.weight[0,0] # <- this is my attempt at weight-sharing
    self.out_layer = nn.Linear(dim_hidden, dim_out)

```

Is there anyway to overcome this? I’d like to have build up clusters of shared weights if this is possible?

---

<div class="post-metadata">

**Author:** ![adamr](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adamr/32/27859_2.png) [@adamr](https://discuss.pytorch.org/u/adamr)\
**Post date:** [August 16, 2020, 11:01am UTC](https://discuss.pytorch.org/t/single-weight-sharing-across-a-network/67798/2 "2020-08-16T11:01:46Z")

</div>

I am not sure if you can share _part_ of the tensor. Try [How to use shared weights in different layers of a model](https://discuss.pytorch.org/t/how-to-use-shared-weights-in-different-layers-of-a-model/71263) and see if that works if you apply it to your case. I am curious too.