Basically, in BatchNorm2D layer, running_mean and running_var are calculated by moving average of all seen batch_mean and batch_var.
However, i want to only see just few batch statistic window to generate running statistic.
For example, running statistics at step 5 are calculated by moving average of batch statistic at only steps 3, 4, 5. Also, batch statistics at only steps 8, 9, 10(Not all steps 1~10) are involved in running statistics at step 10.