Weird artifacts with Super Resolution

I have made a Super Resolution model and it is learning pretty well but for some reason these artifacts appear. It seems like some pixels that should be white are mostly yellow and some pixels that should be black or dark are mostly blue. Does someone know what can cause these artifacts to appear?