Image normalization for UNET with resnet pretrained encoder

Working on healthcare image datasets what should be the normalization stats used on these images, imagenet stats or stats from the new dataset being trained on?

Does it even matter which we choose?

I would expect the custom stats to work better especially if the actual data stats differ between the medical images and ImageNet, but would also recommend to test both approaches.