The documentation for track_running_stats in BatchNorm2D specifies that setting track_running_stats=False will results in stats being computer during runtime as well.
Does this mean that the result will be dependent on the specifics of each batch, even if I call model.eval()? And how does this play together with the affine parameters of the BatchNorm2D layer?
Yes, if you don’t track the running stats, the batch statistics will be used during train and eval, which also means that your result will be dependent on the current batch.
The affine parameters are independent of the running stats and will still be learned.