Component-wise learnable addition to the incoming data


I need something like CAdd in torch:

So it could be Linear (y = Ax + b) but without learnable A.

How can I do it via pytorch?


Yes, you just need to do usual math operations and it will work just fine. For example

weight = nn.Parameter(torch.rand(4))
input = Variable(torch.rand(4))

output = input * weight # weight is a learnable parameter
1 Like