I have a fully connected layer, I want to multiply first half of weights by a number `K`

that has itself to be learnt.

As in suppose the weight of fully connected network is `W [mxn]`

and I have a another parameter K that is initialized to say 0.5.

I want to perform `W[:m//2,:] = W[:m//2,:] * K`

and I need K to change as well

Is there some way to do this, will backpropogation work if I multiply two parameters???

Modifying parameters this way doesn’t work.

In my opinion the best way to do this is to create your own layer using W and K as parameters and using `torch.nn.functional.linear`

in the forward.

for the multiplication, (provided m is even), I’d probably suggest

```
mul = torch.ones(2, 1, 1, device=K.device)
mul[0, 0, 0] = K
weight = (W.view(2, m//2, -1) * mul).view(m, -1)
```

and then using weight. Backpropagation should work this way.

There is a somewhat elaborate way to replace parameters with calculated quantities, see e.g. the spectral norm implementation for inspiration. Unfortunately, there doesn’t seem general interest for a more generally applicable interface (I’ve tried to pitch my ideas at https://github.com/pytorch/pytorch/issues/7313 ).

Best regards

Thomas

Oh thanks!

Functional approach seems cool I guess, bit not neat but good enough

Thanks

Hi, in your example use of `mul`

is to convert parameter into tensor before multiplication right??

EDIT : seems it doesnt make a difference with or without it

Oops, the code example missed the top line because I left it on the triple backtick line :/.

The mul is target the first half only.