Super_resolution checkerboard artifacts

vision
1

The super-resolution example given in:

just works on a single color channel. I tried to adapt it to work on RGB images, w/ the following changes to the network structure:

...
        self.conv1 = nn.Conv2d(3, 64, (5, 5), (1, 1), (2, 2))
        self.conv2 = nn.Conv2d(64, 64, (3, 3), (1, 1), (1, 1))
        self.conv3 = nn.Conv2d(64, 32, (3, 3), (1, 1), (1, 1))
        self.conv4 = nn.Conv2d(32, 3 * upscale_factor ** 2, (3, 3), (1, 1), (1, 1))
...

but I’m getting extreme artifacts, so something’s clearly wrong here:

Original on left, SR on right.

Anyone have any ideas?

Thanks
Ben

EDIT: On closer inspection, it looks like there are checkerboard issues even in greyscale – so maybe the fix for the artifacts is independent of the color issue?

1 Like
2

Yes, this is a common problem with deconvolution / pixel shuffle, and it is still an open question how to best solve this problem.
For some more information on the problem, see http://distill.pub/2016/deconv-checkerboard/