How can i add the output of the embedding layer?

i want to add or take the mean of the embedded vectors after this
self.embeddings = nn.Embedding(vocab_size, embedding_dim)
OR add a linear transformation to every embedded vector (with shared weights of course).

For the first question:

embs = self.embeddings(input)
mean_embs = embs.mean(1).squeeze(1)

For the second question:

embs = self.embeddings(input)
trans_embs = self.linear(embs.view(-1, embedding_dim)).view(embs.size(0), embs.size(1), -1)
1 Like