My data is normalized, each of them is a number between 0 to 1. When I print out the weight, it is a set of number between -1 to 1. But why? Do weight must be number between -1 to 1?
If you don’t apply any constraints on the parameters, they are not necessarily in the range [-1, 1].
E.g. nn.Linear layers are randomly initialized using this method and thus the initial range might be smaller.
1 Like