How to handle Out-of-vocabulary token in inference using torchtext Field?

nlp
2

There is a similar issue here but with a whitespace being looked up: Out-of-vocabulary KeyError on vocab.stoi in TorchText
Even if the comment there is correct – that a whitespace shouldn’t really be part of your vocabulary – shouldn’t it be mapped to the unknown token by default? Unfortunately, they’re not really answering the question about oov words. :sweat_smile:

Issue without much discussion: https://github.com/pytorch/text/issues/337
… Maybe you can revive the issue or create a new one.