Consider convolutional layers. They create filters and pass them through an input, according to its stride and padding parameters. The filter size is the kernel size.
Now, consider a new form of linear layer, like the convolutional layer. Each node is connected to a specific area in the image, like the kernel. Nodes look at sensing regions with ‘stride’ apart. I would like to create such a linear layer, not fully connected and very similar to a convolutional layer. How could I do it? Are there a library you could refer me to? Thank you,
Use F.fold to get the image as the patches just like for convolutional layers (it will be of shape batch x number of patches x size of a patch, then you implement your operation, and F.unfold to get back. The documentation (either for the functional or the modular interface) has an example for convolution.
Best regards
Thomas
1 Like
fold/unfold could be a start if I was trying to pass patches through the same linear layer, however patches have to pass through different weights, unlike cnn where filter weights are shared. Do you have any further ideas?
Thank you for your reply,
This forum can be less full service than you might expect, but:
unfold/fold will get you the patches. If you then reshape to batch x (number_of_patches * patch_size) you can apply linear with different weights.
The technical term for this type of layer is “locally connected layer” by the way.
Best regards
Thomas
1 Like