# How to create an optimizer given a loss function

I’m quite confused about how the optimizer created. From the Pytorch tutorial, when people implement SGD by defining a custom module first, which contains a constructor and a forward function, then using SGD optimizer to calculate the model parameters (please see the following code). My question is what if I am only given a loss function(please see the following pic) and I don’t have a “forward” function, how to create an optimizer? If my question is not clearly stated, please let me know and I’ll provide further information, thanks for any help in advance.

import torch

class model(torch.nn.Module):

``````def __init__(self, D_in, H, D_out):
super(model, self).__init__()
self.linear1 = torch.nn.Linear(D_in, H)
self.linear2 = torch.nn.Linear(H, D_out)

def forward(self, x):
h_relu = self.linear1(x).clamp(min=0)
y_pred = self.linear2(h_relu)
return y_pred
``````

model = model(D_in, H, D_out)

optimizer = torch.optim.SGD(model.parameters(), lr=1e-4)

The optimizer just stored the references to the passed parameters and uses their `.grad` attribute to perform the update step.
It does not depend on the forward pass.
If you are able to generate the gradients in some way, the optimizer will just use them via their `.grad`.

So basically what you mean is that I can directly call “torch.optim.SGD(parameter.grad, lr = 1e-4)” something like that?

No, you still need to pass the parameters as:

``````torch.optim.SGD(model.parameters(), lr=1e-4)
# or as a list
torch.optim.SGD([param1, param2], lr=1e-4)
``````

The optimizer will then use the `param.grad` attributes to update them.

Thank you for your kindly help! It helps me to understand Pytorch a lot!