what is xavier_uniform_, I do understand it is similar to Gaussian distribution but what is the difference between xavier_uniform_ and gaussian distribution
from below code it picked it up
def init_conv2d(self):
“”"
Initialize convolution parameters.
“”"
for c in self.children():
if isinstance(c, nn.Conv2d):
nn.init.xavier_uniform_(c.weight)
nn.init.constant_(c.bias, 0.)
Xavier initialization is a way to initialize the weights with the right values.(They are not big as well as not small.) You can find a detailed explanation from andy’s blog.