nn.Linaer with bias=True is initializing the bias using uniform:
init.uniform_(self.bias, -bound, bound)
I usually override it by initializing it to zeros like so:
fc = nn.Linear(in_dim, out_dim, bias=True)
nn.init.zeros_(fc.bias)
Is there any benefit to the default uniform initialization?
Thank you in advance
Roy