Running UNET architecture on PyTorch gives concurrent black and white circles in output

vision
1

test2
test21152×1024 151 KB

I am using UNET architecture with PyTorch where I have 7 input channels, 3 for cloth, 1 for mask of the same cloth and 3 for densepose of the body.

  • The top row images are the inputs, 3 channel cloth, 1 channel mask of the same and 3 channel body dense pose from left to right.
  • The bottom row rightmost image is my ground truth which should be predicted as it is dependent on the above 3 inputs.
  • The bottom row leftmost image is the output obtained after concatenating the 3 inputs (7 channels) in a UNet model in PyTorch. This is where I get concentric circles at random spots for some reason.
  • The bottom row middle image is the sigmoid of the output and because of the concentric circles there is dark spot on the same and the loss never converges with respect to the ground truth.

I tried changing the input to blank white image, changing the input channels to just a single white image, changing the ground truth and with smaller data but I always get these concentric circles. There was a weight initialization missing in my architecture but now that has been done but even after trying everything, can’t get rid of these concentric circles. Can somebody suggest me the reason why these are occurring or what other parameters I can change?

2

Usually I would recommend to scale down the problem a bit and check for possible bugs in the code.
Since your model is apparently not learning the target (I’m not sure what it represents), I would suggest to use just a small part of your training dataset and try to overfit your model.
If your model successfully learns to predict e.g. 1 or 10 samples, you could try to carefully use more samples etc. and try to find out what goes wrong at which point.
However, if your model can’t even predict a single sample, there are most likely some issues in the code or e.g. the architecture is not suitable for this problem.

3

I have already tried with smaller examples, I even tried changing my input to a single blank white image of 1 channel, I still got concentric circles, only at different spots. Can you suggest me something which I can try changing wrt UNET architecture because that seems to be the only issue right now. Also, I have edited the post to state what are these images and what I have tried. Kindly, have a look, thanks!

4

Thanks for the update.
Could you post a link to your implementation so that I could have a look at your model?