How to implement (nueral net inside neural net) in Pytorch?

I have input data (images) that come with obscene number of channels (700+). To address this, I intend to have conv filter with (1x1) size so that it acts on a single pixel. what I want is: this filter acts only on this pixel (no spatial scanning!).

PS: I read about the “groups trick”, I think it is useful too and I used it in the first layer. But I completely forgot that I still have the problem of 1x1 filter acting on all pixels.

    self.torso = nn.Sequential(OrderedDict([
        ('nn_in_nn', nn.Conv2d(in_channels=ch_in, out_channels=64 * ch_out_multip, kernel_size=(1, 1), stride=1, padding=(0, 0),
                               groups=8 )),

I think I got it, make filter size = size of the image, thus, there’s no scanning, each set of weights from the filter will act only on corresponding pixel, only.

But now I think this is too much parameteres, and I really want to do depth-wise convolution to save some parameters. Groups is not helping at all, it is just separating the filters, no real depth-wise conv is taking place. How can I do that?

I got it, reshape the data to be PIXELS X CHANNELS (i.e. image has one axis only). Next, use 1D convolutions that operate on channels axis as required. To inforce single pixel requirement, use the group trick in 1DConv. (now pixels are thought of as channnels).