Image Denoising using Unet

I’m new to computer vision and deep learning. I’m trying to train this Unet model GitHub - kevinlu1211/pytorch-unet-resnet-50-encoder with Resnet50 as encoder. I want to implement it in a way that I pass two rgb images which are first processed by resnet50 and then the layers are concated before being passed to the decoder. I tried doing it and changed n_classes in the code to 3 to output a 3 channel rgb image just like the inputs but it gives me a distorted image

which I don’t understand why. Please help me with this