Softmax/log_softmax in CTC loss

The docs to suggest using of logarithmized probabilities for an input of CTCLoss

However, there is example in pytorch repo (the DeepSpeech model) where softmax func (instead of log_softmax) is used just for evaluation but not training.

So I want to clarify what should I use for training and evaluation in CTCLoss:

  • softmax/log_softmax for train/eval?
  • identity for the training and softmax/log_softmax for eval like in example I shared above?

As far as I know, for training you need log_softmax. For inference you can just do argmax. But using argmax might only give you Top-1 accuracy. If you use softmax and get top 5 scores you can get Top-5 accuracy. I think DeepSpeech model does something similar.

Thank you for the reply.

So for the training I need to use log_softmax it’s clear now. For the inference I can use softmax to get top k scores.

What isn’t clear is that why DeepSpeech implementation is not using log_softmax in the repo? I suppose there should be an explicit call of log_softmax in the model definition or the model calling, right? Or did I miss something?

model definition

model calling