How are layer weights and biases initialized by default?

2

Linear layers are initialized with

stdv = 1. / math.sqrt(self.weight.size(1))
self.weight.data.uniform_(-stdv, stdv)
if self.bias is not None:
    self.bias.data.uniform_(-stdv, stdv)

See also here.

48 Likes