NN training for function estimation

reinforcement-learning
1

Hello guys,

I’m a pretty new learner in ML/AI. This domain is so amazing but no so easy. In order to learn and understand I’m trying to do my own exercise to learn how Pytorch is working.
To do so I created a test where I want the model to predict the function f(x) = 2x. Pretty simple I guess. But my first attempt to write a python script give me not so bad results.

Here is the script :

import torch

" Data for training"
x_train = torch.FloatTensor([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y_train = torch.FloatTensor([2, 4, 6, 8, 10, 12, 14, 16, 18, 20])

" Data for evaluation of model"
x_test = torch.FloatTensor([2, 4, 6, 8, 10, 12, 14, 16, 18, 20])

model = torch.nn.Sequential(torch.nn.Linear(10, 3),torch.nn.ReLU(),torch.nn.Linear(3, 10),)

loss_fn = torch.nn.MSELoss(reduction='sum')

learning_rate = 1e-4
for t in range(48500):
    y_pred = model(x_train)
    loss = loss_fn(y_pred, y_train)
    if t % 1000 == 999:
        print(t, loss.item())
    
    model.zero_grad()
    loss.backward()
    with torch.no_grad():
        for param in model.parameters():
            param -= learning_rate * param.grad
            
y_pred_to_validate = model(x_test)
print(y_pred_to_validate.detach().numpy())
type or paste code here

And the result displayed is not so bad :

[ 4.138885  8.290704 12.023239 16.034138 19.773151 24.239693 28.20026
 31.853294 35.714657 39.374302]

I would like to have your thoughts and your advises are welcome. To be honest the model I created I take it from another example found on the net but I don’t really understand if it’s well suited for this case or not. It’s the same for the number of iteration, I put 48500 because it gave me a good result and I know that overfitting is not really recommended.

I like criticism, it’s the best way to improve myself. Thank you for your time and long life to PyTorch.

David.

2

Do not hand craft the gradient descent part, use torch.optim
And your model is strange, use one single torch.nn.Linear(10, 10) is more than sufficient for such a simple function.

3

Thank you for your reply. You’re right about the gradient descent. I will try to use the optim to see the difference.

I agree also with you about the model. Related to the training test I choose yes we can use a simpler model but from what I read the linear/relu/linear is a kind of all purpose model. Is it correct?

4

Yes, and in your case, your result looks ok. How to choose your model depends on your application and is totally up to you.

5

Hello guys,

I have 1 question about the part of the for loop code. We are iterating several time to adjust the parameters to fit the model. Using the ML words, can we say it is reinforcement or not? If not what is the difference between reinforcement and this adjustment loop?

Thanks for your help.

6

I personally think the defining feature of reinforcement learning is the bellman equation, bellman equation with discount < 1 gaurantees that you will have a converging result.

You code does not show that you are using this methodology, therefore I don’t think it is reinforcement learning (personally).