Init Weights with Gaussian Kernels

hey,
is there a quick way to fill filters with Gauss filters + noise. i am familiar with …

def weights_init(m):
    if isinstance(m, nn.Conv2d):
        m.weight.data.normal_(0, 0.02)
        m.bias.data.normal_(0, 0.001)

However I don’t know how to init s.t. we have gaussian filters.

Generate your filter with https://docs.scipy.org/doc/scipy-0.16.1/reference/generated/scipy.ndimage.filters.gaussian_filter.html

copy over into m.weight.data
m.weight.data is of shape: nOutputChannels x nInputChannels x kernelHeight x kernelWidth, so you have to generate nOutputChannels * nInputChannels, then make a numpy array of the same shape as m.weight.data and then copy:

generated_filters = ... # some scipy / numpy logic
m.weight.data.copy_(torch.from_numpy(generated_filters))
6 Likes

I solved it as above answer as

        generated_filters = gaussian_filter(m.weight.data, sigma=0.5)
        m.weight.data.copy_(torch.from_numpy(generated_filters))

Thank you.

Gaussian is another word for normal distribution, so you can just use:

torch.nn.init.normal_(m.weight, 0, 0.5)

Assuming you want a standard deviation (or sigma) of 0.5 and a mean of 0.
Also see: torch.nn.init — PyTorch 1.8.1 documentation