I would like to ask your help with an implementation. I am trying to train 200 regressors on the same data ( think of these as 200 copies of the same model and same X, but different Y values to regress to for each).
My future implementation would involve aggregating these 200 predicted values. But for now, I just want to run 200 copies of models, all having the same input but different Y_true to regress to. Also, I do not want them to share weights i.e. each should independently update its weights.
To create this, I tried instantiating multiple instances of the net as follows:
class NeuralNetwork(nn.Module): def __init__(self): super(NeuralNetwork, self).__init__() self.layer1 = nn.Linear(X.shape, 100) # self.layer2 = nn.Linear(100, 10) self.layer3 = nn.Linear(10, 1) def forward(self, x): x = F.relu(self.layer1(x)) x = F.relu(self.layer2(x)) x = self.layer3(x) return x models =  #to store instances of NNs, one per regressor losses =  #to store regressor-wise losses y_pred =  #to store the regression predictions models =  losses = *200 y_pred = *200 for i in range(200): # Initialization loop m = NeuralNetwork() optimizer = optim.Adam(m.parameters(), lr=0.005) mse_loss = nn.MSELoss() models.append(m) for i in range(200): # Execution loop for epoch in range(10): predicted = models[i](x_train_tensor) y_pred[i] = predicted loss = mse_loss(predicted,y_train_tensor[i]) losses[i] = loss loss.backward() optimizer.step()
I have 3 questions about this implementation:
a) Would this work?
b) Is the execution loop the right place to call optimizer.step()?
c) How can I parallelize these regressors? I was reading up on Dataparallel module, but that seems to be for models training on different Xs itself. Could you please share some ideas on this?