The derivative for 'running_mean' is not implemented

Trying to update the batchnorm.running_mean to a new value during training, where new_mean = old_mean * x + b

Seems like batchnorm parameters are not trainable and cannot be manipulated - is there a workaround to this?

Because it’s a running_mean it’s updated as an moving average between the current estimate of the mean and the old value rather than being done via 2 tunable parmeters. It’s explained within the docs BatchNorm1d — PyTorch 1.9.1 documentation

Yeah, I am trying to get it to update with tunable parameters though, wondering if this is possible at all

Quite possibly, you could create a nn.Module class with learnable parameters and just defined the forward pass to be the same BatchNorm? That might work?