Is anything in transformers implementation specific to natural language modeling? Especially, wrt nn embedding.
Not really (see also: Vision Transformers [2010.11929] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale ).
Thanks. I didn’t phrase my question correctly. I mean whether the pytorch implementation is specific to language modeling using transformers. Or nothing is limiting to use it for tabular data.