Where weight initialised in the Pytorch tutorial. In the self.conv1 = nn.Conv2d(3, 6, 5) layer, there are 6 different kernels (output-feature) but how the value of the kernel assigned? is it weight initialised in criterion = nn.CrossEntropyLoss(). any comment will be appreciated.
The weight initialization happens here:
1 Like
@InnovArul thank you for the link. I still cannot understand how weight initialization happened. For example in the Tutorial CIFAR10 could you please tell me where weight initialization happened. I know in the first layer which is conv1 we use 6 filters with the kernel size of 5*5, but my question is how the values of these kernels specified?
They are initialized at the time the Module is instantiated, i.e. conv1 = nn.Conv2d(3, 6, 5), through the reset_parameters() method, which is executed as part of the Module’s __init__. The dimensions of the kernel and bias are set through the module’s constructor arguments. If you want to change the parameters’ initialization you can do so explicitly with the functions defined in torch.nn.init and the Module’s apply(fn) method, e.g.
def init_weights(m):
if type(m) == nn.Linear:
torch.nn.init.xavier_uniform(m.weight)
elif type(m) == nn.Conv2d:
# Some other initalization
net = Net()
net.apply(init_weights)
1 Like