Trying to update the batchnorm.running_mean
to a new value during training, where new_mean = old_mean * x + b
Seems like batchnorm parameters are not trainable and cannot be manipulated - is there a workaround to this?
Trying to update the batchnorm.running_mean
to a new value during training, where new_mean = old_mean * x + b
Seems like batchnorm parameters are not trainable and cannot be manipulated - is there a workaround to this?
Because it’s a running_mean
it’s updated as an moving average between the current estimate of the mean and the old value rather than being done via 2 tunable parmeters. It’s explained within the docs BatchNorm1d — PyTorch 1.9.1 documentation
Yeah, I am trying to get it to update with tunable parameters though, wondering if this is possible at all
Quite possibly, you could create a nn.Module
class with learnable parameters and just defined the forward pass to be the same BatchNorm? That might work?