In the nn.Linear function, do the operations go in the following order. Linear combination, then bias is added…and then if we want to add a non-linearity the non-linearity is added after the addition of the bias. Is this order of operation correct
According to the documentation:
Applies a linear transformation to the incoming data: y=xAt+b
Therefore yes, firstly you apply the transformation and then the bias. Thus, you would apply the non-linearity after the bias.