Using kaiming initialization

For my current neural network I’d like to initialize my model’s weights with the torch.nn.init.kaiming_normal, so I have tried this :
but I receive this error message :
Fan in and fan out can not be computed for tensor with less than 2 dimensions
Is the less than a strictly less than ? I am not sure I understand why it’s failing, if someone could give me some pointers that would be great.

1 Like

If you put print( before the kaiming init, what does it show?

It shows this :

torch.Size([200, 1000])
torch.Size([200, 400])
torch.Size([100, 200])
torch.Size([2, 100])

I’m guessing the last one is the one causing problems then because it’s one dimensional ?

What type of layer is that last one from?
I am guessing that it is a layer of a type that you can’t use kaiming init on.

It corresponds to a BatchNorm layer, so I am guessing you are right about the kaiming initialization not being possible on that. I should have thought about it ^^. Thank you for your help with that!