I’m looking for an operator to compute averages of vectors given a matrix and a list of offsets:
input = Variable(torch.LongTensor([[1,1,1], [2,2,2], [3,3,3], [4,4,4], [5,5,5], [6,6,6]])) offsets = Variable(torch.LongTensor([0,4])) out = my_operator(input, offsets, mode='mean') #Variable containing: # 2.5 2.5 2.5 <- the four first vectors are averaged together # 5.5 5.5 5.5 <- the last two are averaged together #[torch.FloatTensor of size 2x3]
nn.EmbeddingBag module which can take a list of indices and offsets and return a vector of averaged embeddings. One could rewrite this module using the previously defined operator:
def EmbeddingBag(emb, input, offsets, mode='mean'): embedded_input = emb(input.unsqueeze(0)) return my_operator(embedded_input, offsets, mode=mode) input = Variable(torch.LongTensor([1,2,4,5,4,3,2,9])) offsets = Variable(torch.LongTensor([0,4])) emb = nn.Embedding(10, 3) EmbeddingBag(emb, input, offsets)
Is there a way to get this operator? Is it available in pytorch ? in torch ? I feel it could be very useful in many situations.