Segmentation of pure noise


I am working on an image manipulation detection task by detecting the different types of noise in the manipulated region in comparison to the pristine region. At the moment the model I use gets as input an image and outputs the image noise in a way that pristine and manipulated regions can be distingushed visually (see images at the bottom). Now I want to separate/segment the regions and hoped that I can achive better results by using image segmentation nets instead of classical methods. Up until now I tried to use a UNet for this task. However, the dice score, F1 score or MCC score doesn’t even get above 2e-3 and I tried already a lot of different settings. So the best is probably to try a different segmentation model. But before I do this and maybe achieve no results too, I was wondering:
Are the segmentation models used in practice even viable for my task? Most/All of them were design to segment objects and my data is from a completly different domain, so maybe none of them will work? What are your thoughts on this? Are there any segmentation models that could be good for segmenting pure noise?

The manipulated image:

The enhanced noise:

A closeup of the noise structure (Left is noise from the manipulated region, right from pristine region):

Thanks for your input on this matter in advance!