How to initialize the conv layers with xavier weights initialization?

1

Hi, I want to know how torch.nn.init.xavier_uniform is used. I’ve tried to apply it on net.parameters but got the following errors:

params = list(net.parameters())
nn.init.xavier_uniform(params,gain=nn.init.calculate_gain(‘relu’))

Traceback (most recent call last):
File “/Users/elaine/Desktop/YL-Net/train.py”, line 23, in
nn.init.xavier_uniform(params,gain=nn.init.calculate_gain(‘relu’))
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/torch/nn/init.py”, line 205, in xavier_uniform
fan_in, fan_out = _calculate_fan_in_and_fan_out(tensor)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/torch/nn/init.py”, line 165, in _calculate_fan_in_and_fan_out
dimensions = tensor.ndimension()
AttributeError: ‘list’ object has no attribute ‘ndimension’

Thanks!

1 Like
2

This is how it is usually done:

linear1=torch.nn.Linear(N_FEATURES, hiddenLayerSize, bias=True) 
torch.nn.init.xavier_uniform(linear1.weight)
17 Likes
4

This is what I’m looking for. Thank you so much!

5

What it used for Please