Number of input channels in pytorch

1

hello all
I have a train dataset of EEG signals. with the wavelet transform I change the signals to color images.every 16 color images is related to the one task.how can I feed every 16 color images together the input of the convolutional neural network at the same time?
thank you

2

Do you have 16 images in a batch, where each sample is associated with a single class?
Or does each image have 16 channels and you would like to split it somehow?
Could you explain your use case and post some dummy code which shows how your model should work?

Generally, nn.Conv2d expects an input of [batch_size, channels, height, width].

3

if I understand you correctly, you can build a new model such as follows

class SampleModel(nn.Module):
    def __init__(self, num_channel_in=16*3):
        super(SampleModel, self).__init__()
        self.c1 = nn.Conv2d(num_channel_in, 128, kerrnel_size=3, stride=1, padding=1)
        self.c2 = nn.Conv2d(128, 256, kerrnel_size=3, stride=1, padding=1)
        ...
4

thanks for your answer
unlike the gray scale and color images which have 1 and 3 channels respectively. we have 16 color images with three channels. how can I fuse 16 images and create one single image with 16 channels.how can CNN handle images with multiple channels(more than one or three channels)? if that is possible how can I put the images in the train , validation and test folders?the input of the CNN must be just image or it can be an array of numbers?

5

thanks for your answer
unlike the gray scale and color images which have 1 and 3 channels respectively. we have 16 color images with three channels. how can I fuse 16 images and create one single image with 16 channels.how can CNN handle images with multiple channels(more than one or three channels)? if that is possible how can I put the images in the train , validation and test folders?the input of the CNN must be just image or it can be an array of numbers?

6

The channel dimension of CNN can be any size. All of the popular models such as Resnet, VGG have 3 channel of input because all of them get RGB images as a input.
You can build a new model which handle a input that has 16 or 48 channels. There is a very sample model as follow for example.

class SampleModel(nn.Module):
    def __init__(self, num_channel_in=16*3, class_num=None):
        super(SampleModel, self).__init__()
        self.c1 = nn.Conv2d(num_channel_in, 128, kerrnel_size=3, stride=1, padding=1)
        self.c2 = nn.Conv2d(128, 256, kerrnel_size=3, stride=1, padding=1)
        self.c3 = nn.conv2d(256, 512, kernel_size=3, stride=1, padding=1)
        self.gap = nn.AdaptiveAvgPool2d((1, 1))
        self.linear = torch.nn.Linear(512, class_num)

    def forward(self, x):
        x = self.c1(x)
        x = self.c2(x)
        x = self.c3(x)
        x = self.gap(x)
        x = torch.flatten(x, 1)
        x = self.linear(x)
        return x

You can not use standart Imagefolder dataset format. You can write new dataset class and fuse your images in getitem function.

7

thanks for your help
may you explain more about how can I fuse 16 images to one image?

if I shape an array with 16 channels does it work in pyotch as input?

does pytorch give image as input or can I input array of numbers in the train and test folders?

thanks in advance

8

for the second question, yes pytorch can work with 16 channels.

for the third question;
read 16 grayscale image and concatenate them and convert tensor. if your image size is 512 x 512. your concatenated input size 16 x 512 x 512.
u can write custom dataset class for reading images and make them 16 channels input.
Check the link for writing custom dataset.