You can find the different loss functions in the docs.
I assume you’ve already found the PyTorch tutorials and would like to get started creating your own models.
There are a lot of good resources to learn more, e.g. Stanford’s CS231n for Visual Recognition (with free Lecture videos), fast.ai’s course (they use a high-level wrapper built on top of PyTorch) or Andrew NG’s coursera course.
For the beginning you could stick to the following (this is my biased opinion and the recommendations might not be the best for your use case!):
- For regression, try
nn.MSELoss()and no non-linearity for your model output. Also normalizing the target to[0, 1]or[-1, 1]might help. - For classification use
F.log_softmax+nn.NLLLossor no non-linearity +nn.CrossEntropyLoss. - Try
optim.Adamas the default optimizer. - Try
nn.ReLUas your default non-linearity between layers.
Once your models converge you can tweak your code in a fancy way and e.g. use skip connections, cyclic learning rates etc. The deeplearningbook is also a great resource.