Border Artifacts when generating RGB images using a UNIT encoder-decoder architecture

I am using the UNIT architecture (GitHub - milesial/Pytorch-UNet: PyTorch implementation of the U-Net for image semantic segmentation with high quality images) as an encoder-decoder for 3 RGB output images. The issue is that the borders of the output images have color issues as can be seen in the attached image.

I am using a 6-channel input (two images concatenated together) to generate a 3-channel output.

do you crop your images in any way?

I have not cropped the images, the norm for face images is that they are aligned so that face features can be correlated. I will try out random cropping and see how it affects the results. Thanks :slight_smile:

Pls let me know whether this solves the issue. Do you need to do image segmentation? If not or if you are open to different approaches, you can try this: StyleALAE. If you pick the right size of the latent space, you can rather easily do image segmentation, too. If you are keen, I can send my/our implementation of the above presented paper with a little add on so that the latent space is more disentangled.

Thanks @nwn
I’ll try out the method sometime this week. I’ll take a look at the paper and let you know if I can adapt the method for my gains.