Feed RAW or PNG images to neural network

vision
1

Hi,
I have RAW images from a DSLR camera Canon (image.CR2). How can I feed my image into a neural network with input raw images of size 192X192x4 perhaps in .PNG format?

My images are 6000X4000, RGB, RAW file (.CR2).

Of course I need to convert my image in .PNG format, and I did, I tried to resize in 192X192 (but i don’t want to do that to not loose information, so I think would be better to split one image into severale parts of 192X192 size).

But, in any case, when I feed a 192X192 image, the error is this:

“Traceback (most recent call last):
File “C:\Users\ingen\Documents\raw2rgbnet\test-full.py”, line 91, in
output = infer(img)
File “C:\Users\ingen\Documents\raw2rgbnet\test-full.py”, line 71, in infer
output = model(im_pad_th)
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 489, in call
result = self.forward(*input, **kwargs)
File “C:\Users\ingen\Documents\raw2rgbnet\models\full_mix3_deep_encoder_decoder.py”, line 141, in forward
fix_s = self.fix_path(fix_s)
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 489, in call
result = self.forward(*input, **kwargs)
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\container.py”, line 92, in forward
input = module(input)
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py”, line 489, in call
result = self.forward(*input, **kwargs)
File “C:\Users\ingen\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\conv.py”, line 320, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [32, 4, 3, 3], expected input[1, 3, 192, 192] to have 4 channels, but got 3 channels instead”

Maybe the problem is the 4th channel?
If you need some code, ask me.

Thanks.

2

You could try to use e.g. rawpy to open the RAW images without transforming them into a particular (lossy) image format.

The error is raised since your conv layer expects an input image with 4 channels, while your input has 3 channels.
I guess you might expect the image files to have an additional alpha channel?
If that’s the case you should take a look at the image loading and processing and make sure this channel is also loaded.
Alternatively, change in_channels to 3 in the conv layer if you want to work with your current input tensors.

3

Thanks. Actually in this model I don’t know the nature of the additional channel, but in another model where I got nearly the same error, the owner answered me:“input raw images should be 4-channel Bayer images ([R, Gr, B, Gb]), not the 3-channel”, this is criptyc for me.

So, the BAYER CHANNEL is an alpha channel? I mean, if I add an alpha channel and the training is performed with the bayer channel, is it the same?

P.S. I tried to add the alpha channel, but the result (this is an enhancement image code) is an image with a huge PURPLE dominant.

4

There shouldn’t be a Bayer “channel”, but the Bayer filter mosaic is an arrangement of color filters used in digital image sensors.
It should contain a pattern of 50% green, 25% red and 25% blue pixels, thus the color format would be BGGR, RGBG, GRGB, or RGGB.
In that case 4 input channels might be expected. However, your current images seem to only contain 3 channels so you would have to check, if you are really reading the RAW images in this format.

5

I transformed with GIMP the raw format .CR2 in .PNG.
I think the node is here: the bayer channel, so I’m getting some informations about my camera (Canon EOS 2000D) and the bayer. I also thinking to use rawpy to read the images… I keep you update and very thanks for your answer!