Hi there I was trying to use nn.Sequential module in a simple regression problem, just a line. The thing is that I print MSELoss every iteration and it is trying to converge at 1, intead of the values I am wondering.
Here is the code:
NN = nn.Sequential( nn.Linear(1,3), nn.Sigmoid(), nn.Linear(3,4), nn.Sigmoid(), nn.Linear(4,7), nn.Sigmoid(), nn.Linear(7,1), nn.Sigmoid() ) criterion = nn.MSELoss() optimizer = optim.Adam(NN.parameters(), lr=0.01) X = torch.rand(200,1) y = 2*X + 3 print(X.shape,y.shape) for i in range(1000): NN.train() optimizer.zero_grad() out = NN(X) #print(out,y) loss = criterion(y,out) loss.backward() optimizer.step() with torch.no_grad(): print ("#" + str(i) + " Loss: " + str(loss.item()))
If I try this to check the results:
newX =  newY =  for i in range(5): r = random.uniform(0, 1) newX.append(r) newY.append(NN(torch.tensor([r])).item())
[0.9996629953384399, 0.9996600151062012, 0.9996635913848877, 0.9996637105941772, 0.9996615648269653].
I have tryed lot of things changing every on the network but don’t know what is happening.