Intermediate Fusion

1

What is CNN intermediate fusion? How can I develop intermediate fusion in a pretrained AlexNet model in PyTourch?

2

I don’t know what “intermediate fusion” refers to, but in case you are interested in fusing multiple operations into one, you might want to script the model via torch.jit.script or use the newly introduced torch.compile operation in the nightly binaries.

1 Like
3

Thank you for your reply, ptrblck. Feature-level fusion, also known as intermediate fusion. In intermediate fusion, the features, respective to each modality, are concatenated before classification; and intermediate fusion protocols operate on the feature map.

So, I’m looking for feature-level fusion of two CNN models (like Alexnet + Googlenet).

4

class MyEnsemble(nn.Module):
def init(self, modelA, modelB, nb_classes=4):
super(MyEnsemble, self).init()
self.modelA = modelA
self.modelB = modelB
# Remove last linear layer

    self.modelA.classifier = nn.Identity() #densenet201
    self.modelB.fc= nn.Identity() #resnet101
    
    # Create new classifier

    self.classifier = nn.Linear(1920+2048, nb_classes)
    #  mat1 and mat2 shapes cannot be multiplied (16x514 and 512x4096)

========
As in the above code, Removed last linear layer and Create new classifier. Now I want to remove middle layer and want to Create new classifier in this code???how??

5

It depends on the model and might not be possible as explained in your cross-post.