Where can i find nn.transformer examples?

1

All examples i find in google about self made attention, but i wonna use official one.

2

Try

https://pytorch.org/tutorials/beginner/transformer_tutorial.html

Best regards

Thomas

3

Note that this example uses only the encoder. It’ not the full transformer

1 Like
4

Do i get it right and encoder return 2d array like batch_size*features. Can i get 3d array batch_size*position*features like embedding? I wonna try to feed it into simple rnn.

6

The nn.TransformerEncoder returns a 3d tensor with shape [S, N, E] where S is the length of your input sequence, N is batch size, E represents feature dimensions.