How to fuse feature maps to a big feature map for Conv layers

vision
1

Hi guys!

I meet some problems and hope I could get help from you. My problems are as below:

I wanna design a DCNN to integrate feature maps from different DCNNs for classification. My problem is how to fuse these feature maps? My initial idea is to concatenate these feature maps and then put the big feature map to convolutional layers. However, the number of feature maps from different DCNNs is variable. This makes the in_channels of the Con2d is uncertain after concatenating them.

According to the solution to cope with variable image size, I wanna try the AdaptiveAvgPooling. But it seems this solution need to set the parameter of in_channels of Con2d to None, which is impossible. Right?

Is there any other good solutions? Thank you in advance!

2

Your idea of concatenating the feature maps and then pass as a new volume to a conv layer sounds plausible.
Could you explain your concerns a bit and why you have a variable number of feature maps?
Do you mean the number of channels by that statement or the spatial size?
In the latter case, you could use e.g. (adaptive) pooling layers to create activation maps of the same spatial size before concatenating them.