Basic_english and spacy tokenizer

On some of the systems, I need to replace “basic_english” with “spacy”, it works.

What is the difference between these basic_english and spacy tokenizer?

Why do you need to replace it with spaCy? What does not work correctly for you? The basic_english tokenizer should work as long as your language is English. It’s in the name. It will do a very basic string normalization and then splitting by whitespace. If you use other parsers, such as spaCy, that library will be used to tokenize the text.