Hi,
I am Trying to create an effeicient neural net and the best opzimizer + learning rate to take image and tabular data and get a rank (the target) that is a float, the model and optimizer are as following:
Model:
class MyModel7(nn.Module):
def __init__(self):
super(MyModel7, self).__init__()
self.cnn = GoogleNet()# any conv net for image processing
self.dropout = nn.Dropout(0.5)
self.norm = nn.BatchNorm1d(250)
# 12 tabular features
self.fc1 = nn.Linear(12 + 1, 250)
self.fc2 = nn.Linear(250,1)
def forward(self, image, data):
x1 = self.cnn(image)
x2 = data
x = torch.cat((x1, x2), dim=1)
x = F.relu(self.fc1(x))
#apply dropout
x = self.dropout(x)
# apply batchnorm
x = self.norm(x)
x = self.fc2(x)
return x
optimizer:
optimizer = optim.Adam(model.parameters(), lr = 1e-3, weight_decay=1e-3)
the model is trained with an early stopping mechanism to ensure the best epoch parameters are saved
This model is performing ok but i want it to be as best as possible, what is the best way to approach this regression problem with the given data to minimize the MSE?