Problem with Linear Regression

1

Hi all.i am new to pytorch and i am trying to implement linear regression through the following code

import torch
from torch.autograd import Variable
x=[[1.2,1.3],[1.6,1.3],[1.9,1.2],[1.4,1.5],[1.3,1.9]]
y=[[2.5],[2.9],[3.1],[2.9],[3.2]]

x_data=Variable(torch.tensor(x))
y_data=Variable(torch.tensor(y))

class Model(torch.nn.Module):
    def __init__(self):
        super(Model,self).__init__()
        self.linear=torch.nn.Linear(2,1)
    def forward(self,x):
        return self.linear(x)


model=Model()

criterion=torch.nn.MSELoss()
optimizer=torch.optim.SGD(model.parameters(),lr=0.05)

for epoch in range(1000):
    y_pred=model(x_data)

    loss=criterion(y_pred,y_data)

    

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

print(list(model.parameters()))

The parameters of this model should be [[1.],[1.]] and [[0]].

but i am getting incorrect parameters like [[ 0.7835, 0.7870]] ,[ 0.6305]

whats wrong with my code?

4

Do you use bias? Are the predictions correct (despite the strange parameters)?

5

predictions are somewhere around correct…but since the loss function is convex, it should settle down to only one value which is [[1],[1]],[0]…why is that not happening

6

Of what magnitude is the difference between the prediction and the target?

Did you try varying the learning rate/learning rate decay?
It might be possible that the learning rate is simply to large which causes oscillations around the minimum and prevents the optimizer from finding the global minimum.

You could imagine of a valley where you want to reach the ground. If you start at one side and your stepsize is too large you will only be able to get somewhere near the bottom but you will never reach the ground.

7

well the loss is around 0.14 but still the parameters are not correct

8

I tried your code now and it seems to work on my machine with different seeds.
For convergence I just set the number of epochs to 20000 and get the parameters:

Epoch 19999, params [Parameter containing:
tensor([[ 1.0000,  1.0000]]), Parameter containing:
tensor(1.00000e-05 *
       [ 2.9201])]
9

yeah…maybe that was the error…setting number of epochs to a lesser value

Thanks a lot