Do I need to define/register Parameter for each of the Variables in my PyTorch code? or if I just define Linear/etc. it automatically assigns weights and updates the weights during training phase?
A layer derived from nn.Module or an nn.Parameter assigned as an attribute inside a custom model, will be registered as a model parameter and thus will be returned by calling model.parameters().
To optimize these parameters, you would have to pass them to the optimizer.
If you want to train all registered parameters, you could simply use:
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)
On the other hand, you can pass a list with whatever parameters you would like to optimizer.
E.g. you could pass only the parameters of a specific layer as:
optimizer = torch.optim.SGD(model.my_layer.parameters(), lr=1e-3)
Have a look at e.g. this tutorial to learn more about the PyTorch work flow.