What's the default initialization methods for layers?

For PyTorch 1.0, most layers are initialized using Kaiming Uniform method. Example layers include Linear, Conv2d, RNN etc. If you are using other layers, you should look up that layer on this doc. If it says weights are initialized using U(...) then its Kaiming Uniform method. This is in line with PyTorch’s philosophy of “reasonable defaults”.

If you want to override default initialization then see this answer.

13 Likes