Hi! In near future i can start natural language processing project with transformer models, should i use HuggingFace Transformers library or do i write my own version.
It’s going to be used in several separate projects and if i write my own version i can be more free i think.
What is your suggestions, thanks in advance.
(I’m going to use HuggingFace tokenizer, it’s ridiculously fast)
I think it depends how much “flexibility” you need and if you couldn’t achieve it by manipulating Huggingface models.
Rewriting the models sounds like a good way to get familiar with the implementations, but you should consider how long it might take, and check if this time would be better spent by manipulating the models and run more experiments.