RuntimeError: Given groups=1, weight of size [3, 3, 1, 1], expected input[4, 1, 64, 64] to have 3 channels, but got 1 channels instead

vision
1

I am working on the following code:

The code is written for RGB images (3 channels), but the dataset I’m working has only single channel. I have changed the line numbers 108 and 109 of dataset.py file to process the single channel.

But I got the following error:

Traceback (most recent call last):
  File "train.py", line 68, in <module>
    main(cfg)
  File "train.py", line 63, in main
    solver.fit()
  File "F:\CFSRCNN\cfsrcnn_x2\solver.py", line 89, in fit
    sr = refiner(lr, scale)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\parallel\data_parallel.py", line 159, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "F:\CFSRCNN\cfsrcnn_x2\model\cfsrcnn.py", line 74, in forward
    x0 = self.sub_mean(x)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "F:\CFSRCNN\cfsrcnn_x2\model\ops.py", line 29, in forward
    x = self.shifter(x)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\conv.py", line 423, in forward
    return self._conv_forward(input, self.weight)
  File "C:\Users\anaconda3\envs\CFSRCNN\lib\site-packages\torch\nn\modules\conv.py", line 420, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [3, 3, 1, 1], expected input[4, 1, 64, 64] to have 3 channels, but got 1 channels instead

I need to change the model/cfsrcnn.py file to fed the single channel. How to change MeanShift. function to process the single channel. Please help me to change this model code so that I can fed the single channel. The MeanShift function is given below:

class MeanShift(nn.Module):
    def __init__(self, mean_rgb, sub):
        super(MeanShift, self).__init__()

        sign = -1 if sub else 1
        r = mean_rgb[0] * sign
        g = mean_rgb[1] * sign
        b = mean_rgb[2] * sign

        self.shifter = nn.Conv2d(3, 3, 1, 1, 0) #3 is size of output, 3 is size of input, 1 is kernel 1 is padding, 0 is group 
        self.shifter.weight.data = torch.eye(3).view(3, 3, 1, 1) # view(3,3,1,1) convert a shape into (3,3,1,1) eye(3) is a 3x3 matrix and diagonal is 1.
        self.shifter.bias.data   = torch.Tensor([r, g, b])
        #in_channels, out_channels,ksize=3, stride=1, pad=1
        # Freeze the mean shift layer
        for params in self.shifter.parameters():
            params.requires_grad = False

    def forward(self, x):
        x = self.shifter(x)
        return x
2

Change the in_channels argument of the self.shilfter conv layer to 1 and it should work.

3

I have changed the in_channels argument of the self.shilfter conv layer to 1 in the following line ( cfsrcnn/model/ops.py: line 20):
self.shifter = nn.Conv2d(3, 1, 1, 1, 0)

But, the error still persists.

In MeanShift function, they have used mean_rgb variable to calculate r, g, b, but in my case, I have gray images, so how to modify the code according to my dataset.

4

You’ve changed the out_channels to 1 and kept in_channels=3.

5

I have corrected this. But now I’m getting this error:

Traceback (most recent call last):
File “F:\CFSRCNN\cfsrcnn_x2\model\cfsrcnn.py”, line 75, in forward
x1 = self.conv1(x0)
RuntimeError: Given groups=1, weight of size [64, 1, 1, 1], expected input[4, 3, 64, 64] to have 1 channels, but got 3 channels instead

6

@ptrblck Thnak you so much for your solution. The previous problem is solved now. But I have got a new error:

RuntimeError: The size of tensor a (128) must match the size of tensor b (168) at non-singleton dimension 2
The input LR image size is 128x128 and output should be 256x256. The evaluate function returns the sizes of lr, lr_ptach, sr and results are as follows:

lr=torch.Size([1, 128, 128])
lr_ptach=torch.Size([4, 1, 84, 84]) #shave=20
sr=torch.Size([4, 3, 168, 168])
results=torch.Size([1, 256, 256])
How to solve this problem?

def evaluate(self, test_data_dir, scale=2, num_step=0):
        global mean_psnr, mean_psnr1, mean_ssim
        mean_psnr = 0
        mean_psnr1 = 0
        mean_ssim = 0
        #print 'sdfs'
        cfg = self.cfg
        self.refiner.eval()
        test_data   = TestDataset(test_data_dir, scale=scale)
        test_loader = DataLoader(test_data,
                                 batch_size=1,
                                 num_workers=0,
                                 shuffle=False)

        for step, inputs in enumerate(test_loader): #step is the number of test images
            hr = inputs[0].squeeze(0) #reduce the first dimension
            lr = inputs[1].squeeze(0)
            print(lr.size())
            name = inputs[2][0]
            h, w = lr.size()[1:]
            #print lr.size()[1:]
            h_half, w_half = int(h/2), int(w/2)
            h_chop, w_chop = h_half + cfg.shave, w_half + cfg.shave

            # split large image to 4 patch to avoid OOM error
            lr_patch = torch.FloatTensor(4, 1, h_chop, w_chop)
            #rint(lr_patch.size())
            lr_patch[0].copy_(lr[:, 0:h_chop, 0:w_chop])
            lr_patch[1].copy_(lr[:, 0:h_chop, w-w_chop:w])
            lr_patch[2].copy_(lr[:, h-h_chop:h, 0:w_chop])
            lr_patch[3].copy_(lr[:, h-h_chop:h, w-w_chop:w])
            lr_patch = lr_patch.to(self.device)
            print(lr_patch.size())

            # run refine process in here!

            with torch.no_grad():
                sr = self.refiner(lr_patch, scale).data
                print(sr.size())
            h, h_half, h_chop = h*scale, h_half*scale, h_chop*scale
            w, w_half, w_chop = w*scale, w_half*scale, w_chop*scale
            
            # merge splited patch images
            result = torch.FloatTensor(1, h, w).to(self.device)
            print(result.size())
            result[:, 0:h_half, 0:w_half].copy_(sr[0,0:h_half, 0:w_half])
            result[:, 0:h_half, w_half:w].copy_(sr[1,0:h_half, w_chop-w+w_half:w_chop])
            result[:, h_half:h, 0:w_half].copy_(sr[2,h_chop-h+h_half:h_chop, 0:w_half])
            result[:, h_half:h, w_half:w].copy_(sr[3,h_chop-h+h_half:h_chop, w_chop-w+w_half:w_chop])
            sr = result
            hr = hr.cpu().mul(255).clamp(0, 255).byte().permute(1, 2, 0).numpy() #(644.1024,3) is the same dimensional with the size of input test image in dataset.py.
            sr = sr.cpu().mul(255).clamp(0, 255).byte().permute(1, 2, 0).numpy()
           
            # evaluate PSNR
            # this evaluation is different to MATLAB version
            # we evaluate PSNR in single channel
            bnd = scale
            im1 = hr[bnd:-bnd, bnd:-bnd]
            im2 = sr[bnd:-bnd, bnd:-bnd]
            mean_psnr += psnr(im1, im2) / len(test_data)
            mean_ssim += calculate_ssim(im1,im2)/len(test_data)
        print('epochs is %d, mean_psnr is %f, mean_ssim is %f' %(self.step,mean_psnr,mean_ssim))
        return mean_psnr