Component-wise learnable addition to the incoming data

avgaydashenko · April 1, 2017, 1:02pm

Hi!

I need something like CAdd in torch:

torch/nn/blob/master/doc/simple.md#nn.CAdd

<a name="nn.simplelayers.dok"></a>
# Simple layers #
Simple Modules are used for various tasks like adapting Tensor methods and providing affine transformations :

  * Parameterized Modules :
    * [Linear](#nn.Linear) : a linear transformation ;
    * [LinearWeightNorm](#nn.LinearWeightNorm) : a weight normalized linear transformation ;
    * [SparseLinear](#nn.SparseLinear) : a linear transformation with sparse inputs ;
    * [IndexLinear](#nn.IndexLinear) : an alternative linear transformation with for sparse inputs and max normalization ;
    * [Bilinear](#nn.Bilinear) : a bilinear transformation with sparse inputs ;
    * [PartialLinear](#nn.PartialLinear) : a linear transformation with sparse inputs with the option of only computing a subset ;
    * [Add](#nn.Add) : adds a bias term to the incoming data ;
    * [CAdd](#nn.CAdd) : a component-wise addition to the incoming data ;
    * [Mul](#nn.Mul) : multiply a single scalar factor to the incoming data ;
    * [CMul](#nn.CMul) : a component-wise multiplication to the incoming data ;
    * [Euclidean](#nn.Euclidean) : the euclidean distance of the input to `k` mean centers ;
    * [WeightedEuclidean](#nn.WeightedEuclidean) : similar to [Euclidean](#nn.Euclidean), but additionally learns a diagonal covariance matrix ;
    * [Cosine](#nn.Cosine) : the cosine similarity of the input to `k` mean centers ;
    * [Kmeans](#nn.Kmeans) : [Kmeans](https://en.wikipedia.org/wiki/K-means_clustering) clustering layer;
  * Modules that adapt basic Tensor methods :

This file has been truncated. show original

So it could be Linear (y = Ax + b) but without learnable A.

How can I do it via pytorch?

fmassa · April 2, 2017, 1:24am

Hi,

Yes, you just need to do usual math operations and it will work just fine. For example

weight = nn.Parameter(torch.rand(4))
input = Variable(torch.rand(4))

output = input * weight # weight is a learnable parameter