I searced for any books about nlp with pytorch but found only one and it’s not suitable with the updated for torchtext.
I open the documents and followed the classification tutorial and want to apply it into an arabic dataset but when i get into the tokenizer = get_tokenizer(“”) I don’t know what to apply spacy is for english as they see in torchtext and theres is only basic_english I know that i am missing some steps but couldn’t figure it !
Try sentencepiece. It’s a good way to turn a sampling of your text into a fixed number of tokens. You may need some trial and error, but it’s pretty straight forward.