Decoder only stack from torch.nn.Transformers for self attending autoregressive generation

Matias_Vasquez · April 2, 2022, 11:48am

I am not going to claim I know what I am doing here , but I think you can guide yourself with the github repository to see how you can implement the GPT2 class directly.

github.com

huggingface/transformers/blob/60d27b1f152c181705191765661967fef3016cef/src/transformers/models/gpt2/modeling_gpt2.py#L668


      
              model.parallelize(device_map)  # Splits the model across several devices
              model.deparallelize()  # Put the model back on cpu and cleans memory by calling torch.cuda.empty_cache()
              ```
          """
          
          
          @add_start_docstrings(
              "The bare GPT2 Model transformer outputting raw hidden-states without any specific head on top.",
              GPT2_START_DOCSTRING,
          )
          class GPT2Model(GPT2PreTrainedModel):
              _keys_to_ignore_on_load_missing = ["attn.masked_bias"]
          
              def __init__(self, config):
                  super().__init__(config)
          
                  self.embed_dim = config.hidden_size
          
                  self.wte = nn.Embedding(config.vocab_size, self.embed_dim)
                  self.wpe = nn.Embedding(config.max_position_embeddings, self.embed_dim)

Here is a small example to use GPT2 directly and not through their pipeline. This way you can feed tensors directly to the model. Here I am using the GPT that outputs the raw hidden-states without a specific head on top. If you parse through the source code, you will see more with language modeling for single or multiple choice or sequence classification for example.

from transformers.models.gpt2 import GPT2Model, GPT2Config

config = GPT2Config()
model = GPT2Model(config)
print(model.config)

sentence = torch.rand(2, 20, 768)

output = model(inputs_embeds=sentence)

Hope this helps a little bit more!