Are transformers design specific to nlp

1

Hi folks
Is anything in transformers implementation specific to natural language modeling? Especially, wrt nn embedding.

2

Not really (see also: Vision Transformers [2010.11929] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale ).

3

Thanks. I didn’t phrase my question correctly. I mean whether the pytorch implementation is specific to language modeling using transformers. Or nothing is limiting to use it for tabular data.