I have two CNNs where each CNN runs on a different dataset. The output feature vectors of these CNNs (basically without the last fc layers) are concatenated and given to a classifier. How do I train such a network?
How do I train such a network?
what are you exactly looking for? a pointer or a full blown example?
Generally, you can write your autograd expressions in free form and call backward on the final loss of the classifier.
I didn’t realize that my Net class could take 2 inputs and have two parallel networks inside it in free form. Thank you for the help xD
I also want to ensemble two CNNs for jointly training. May you share some code examples?
Here’s a basic example: (sorry for the late reply, was out of town)
class Net(nn.Module): def __init__(self): super(Net, self).__init__() self.pool = nn.MaxPool2d(2, 2) #1st net self.conv1 = nn.Conv2d(3, 64, 3, padding = 1) self.conv2 = nn.Conv2d(64, 128, 3, padding = 1) self.conv3 = nn.Conv2d(128, 256, 3, padding = 1) self.conv4 = nn.Conv2d(256, 256, 3, padding = 1) self.conv5 = nn.Conv2d(256, 512, 3, padding = 1) self.conv6 = nn.Conv2d(512, 512, 3, padding = 1) self.conv7 = nn.Conv2d(512, 512, 3, padding = 1) self.conv8 = nn.Conv2d(512, 512, 3, padding = 1) self.fc1 = nn.Linear(512*9, 4096) self.fc2 = nn.Linear(4096, 4096) #2nd net self.conv1o = nn.Conv2d(1, 3, 2) self.conv2o = nn.Conv2d(3, 3, 2) self.fc1o = nn.Linear(16 * 5 * 5, 120) self.fc2o = nn.Linear(120, 84) def forward(self, x, y): x = self.pool(F.relu(self.conv1(x))) x = self.pool(F.relu(self.conv2(x))) x = F.relu(self.conv3(x)) x = self.pool(F.relu(self.conv4(x))) x = F.relu(self.conv5(x)) x = self.pool(F.relu(self.conv6(x))) x = F.relu(self.conv7(x)) x = self.pool(F.relu(self.conv8(x))) x = x.view(-1, 512*9) #can also do x.view(-1, 1) y = F.relu(self.conv1o(y)) y = F.relu(self.conv2o(y)) y = x.view(-1, 1) y = F.relu(self.fc1o(y)) y = F.relu(self.fc2o(y)) x = F.relu(self.fc1(x)) x = F.dropout(x, training = self.training) x = F.relu(self.fc2(x)) x = F.dropout(x, training = self.training) x = torch.cat((x, y)) return F.log_softmax(x)
Your advice is so instructive to me. However, I wonder about how to copy with the input training set because of the two inputs. Please reply me when you are not busy. Thanks for your attention.
I’m sorry, I didn’t understand your question. Did you mean how to give two inputs into the model? I use two datasets here, one for each sub model.
Is there a full blown example of how to ensemble two models for example- resnet152 and densenet161 for prediction on the same dataset? I would really appreciate it.
I dont have a full-blown example, but really it’s as simple as something like:
for input, targets in test_loader: out_resnet = resnet(input) out_densenet = densenet(input) out = out_resnet + out_densenet loss = loss_fn(out, target) total_loss += loss.item()
Thank you so much @smth. Appreciate your quick response. Is it possible to save these two models in one checkpoint for submission? I am participating in a challenge that requires me to load a saved checkpoint as a part of the submission.
Sure, something like this:
torch.save([densenet.state_dict(), resnet.state_dict()], 'checkpoint.pt') densenet_state, resnet_state = torch.load('checkpoint.pt') densenet = DenseNet(...).load_state_dict(densenet_state) resnet = Resnet(...).load_state_dict(resnet_state)