When should I normalize my image data?

I’m working on a medical image database and I’ve trained a network that works fine on it without applying any normalization. But when I normalize the data the results deteriorate a little bit. Why should I normalize the images? Does it have anything to do with testing on unseen images from a new dataset?

Normalization improves learning efficiency as the input and activation values fall in close ranges.