Minimalistic RoBERTa example?

I’m trying to find a minimalistic RoBERTa example that I can modify and use for some research on protein folding.

So far all I can find is fairseq:


Which definitely have the model I’m looking for, but it also looks like there is tons of other stuff that makes just extracting the model quite complicated.

Does anyone have or knows of a simpler place to get a base RoBERTa model, it shouldn’t be pretrained at all, I just want the base framework.