# How should I add a Gaussian noise to the weights of network?

I’m new in PyTorch. I am trying to write a function that adds some arbitrary Gaussian noise to the wights during the training process. my code is like this

for m in model.modules():
if hasattr(m, ‘weight’):
and my question is the shape of “m” how can I create noise with its shape?
I’m confused how should I do that, any help would be appreciated.

do you mean something like this?

``````with torch.no_grad():
``````

Actually my model is some triple network like below:

class EmbeddingNet(nn.Module):
def init(self):
super(EmbeddingNet, self).init()
self.convnet = nn.Sequential(nn.Conv2d(1, 32, 5), nn.PReLU(),
nn.MaxPool2d(2, stride=2),
nn.Conv2d(32, 64, 5), nn.PReLU(),
nn.MaxPool2d(2, stride=2))

``````    self.fc = nn.Sequential(nn.Linear(64 * 4 * 4, 256),
nn.PReLU(),
nn.Linear(256, 256),
nn.PReLU(),
nn.Linear(256, 2)
)

def forward(self, x):
output = self.convnet(x)
output = output.view(output.size()[0], -1)
output = self.fc(output)
return output

def get_embedding(self, x):
return self.forward(x)
``````

class TripletNet(nn.Module):
def init(self, embedding_net):
super(TripletNet, self).init()
self.embedding_net = embedding_net
embedding_net = EmbeddingNet()
model = TripletNet(embedding_net)

So, I am using define some arbitrary mean and std then use them into make Gaussian noise how can i add this noise during the training process of this network in each epoch. because in each epoch i define a new mean and std.

do you mean sampling from normal distribution with new mean and std, then we could use,

``````normal_dist = torch.distributions.Normal(loc=torch.tensor([0.]), scale=torch.tensor([1.0]))
``````

replace torch.tensor([0.]) with mean value, and torch.tensor([1.0]) with std value
one example would be

``````x = nn.Linear(3, 3)
t = normal_dist.sample((x.weight.view(-1).size())).reshape(x.weight.size())
``````
1 Like

First get the parameters of your model as a vector

``````from torch.nn.utils import vector_to_parameters, parameters_to_vector

param_vector = parameters_to_vector(model.parameters())
``````

Then sample a gaussian noise of the same size as this vector and add it.

``````n_params = len(param_vector)
noise = Normal(0, 1).sample_n(n_params)
`vector_to_parameters(param_vector, model.parameters())`