What is a difference between Loss Function and Evaluation Matrix also why we cannot use evaluation matrix as a Loss functions.
I know it is a pretty noob question.
Thanks.
A loss function is used to train the model and has usually “good” properties to calculate the gradients.
E.g. you could use torch.argmax on out model output, which would give you the predicted label.
However, you wouldn’t be able to train your model with it, as argmax has almost everywhere a zero gradient. That’s why you would fall back to a soft argmax method (softmax) to use this as a proxy loss function.