Normal noise into part of tensor

1

Hello,
I want to input some normal noise into part of tensor of some inner layer, for example, I have hooked the first 16 of a layer tensor, How could I add the noise?
I notice

class AddGaussianNoise(object):
def init(self, mean=0., std=1.):
self.std = std
self.mean = mean

def __call__(self, tensor):
    return tensor + torch.randn(tensor.size()) * self.std + self.mean

def __repr__(self):
    return self.__class__.__name__ + '(mean={0}, std={1})'.format(self.mean, self.std)

is adding noise to orignal input, so can you help me out in the inner layer, part of tensor? Thank you.

2

I’m unsure what “inner layer” means in your context, but in case you want to add noise to an intermediate activation you could write AddGaussianNoise as an nn.Module and apply it in the forward pass or you could also add the noise directly to the activation in the forward method.

3

sorry for my expression, The inner layer what I want to mean is, for example, I have 5 layers NN, and I hook out the least second layer, in this layer I have a total of 200 nodes, and I only want to add noise to the first 16 nodes. Based on this situation, could you help me out? Thank you so much!

4

I’m not sure what “nodes” refer to, but in case it’s the output activation, you could use the previously suggested method of adding the noise directly to the output in the forward method.