Initialize different convolution layers of a network with different method

1

I want to initialize different convolution layers of a network with different methods. Is there any way to do this thing? Please help me.

2

I found a course on Deep Learning from Chinese University of Hong Kong useful. They have a tutorial on PyTorch that shows how we can initialize different layers.

  • Here is the link to the course.
  • Here is the link to the slides for the tutorial.
  • Here is the link to the code that is discussed during the tutorial.
  • Here is the link to the audio for the class session.

Slide number 14 talks about how to initialize parameters. In the audio at 36 minutes in, the instructor talks about this slide. I think this will help.

EDIT-1:
Here is the reply for the Weight Initialization questions that shows how to initialize a layer with Xavier init. Extrapolation from one layer to many layers should be simple.

4 Likes