Hi, I am just a beginner of PyTorch, even a beginner of Python. So this question may too simple for you but I really want to get some help.
I want to train a neural network which is included sub-networks. It looks something like that.
In my opinion, this network could be implemented by constructing three simple fully-connection neural networks.
My implementation is:
#firstly, define the simple NN class Net(nn.Module): def __init__(self,n_input,n_hidden1,n_hidden2): super(Net, self).__init__() self.hidden1 = nn.Linear(n_input,n_hidden1) self.hidden2 = nn.Linear(n_hidden1,n_hidden2) self.output = nn.Linear(n_hidden2,1) def forward(self,x): x = F.tanh(self.hidden1(x)) x = F.tanh(self.hidden2(x)) x = self.output(x) return x #construct 3 same nets net =  for i in range(3): net.append(Net(3,5,5)) # define loss function and optimizer optimizer =  for i in range(3): optimizer.append(torch.optim.SGD(net[i].parameters(), lr = 0.2)) loss_func = torch.nn.MSELoss() #training neural network for step in range(300): #training circles for i in range(3): output[:, i] = net[i](input[i, :, :]) prediction = torch.sum(output,1) loss = loss_func(prediction,target) for i in range(3): optimizer[i].zero_grad() loss.backward(retain_variables=True) for i in range(3): optimizer[i].step()
after several circles, all output value become to the same value.
There must something wrong with my code. Could anyone give me some help?
Or are there any better way to implement this network?
Thanks a lot.