I’m looking for an operator to compute averages of vectors given a matrix and a list of offsets:
input = Variable(torch.LongTensor([[1,1,1],
[2,2,2],
[3,3,3],
[4,4,4],
[5,5,5],
[6,6,6]]))
offsets = Variable(torch.LongTensor([0,4]))
out = my_operator(input, offsets, mode='mean')
#Variable containing:
# 2.5 2.5 2.5 <- the four first vectors are averaged together
# 5.5 5.5 5.5 <- the last two are averaged together
#[torch.FloatTensor of size 2x3]
Considering the nn.EmbeddingBag
module which can take a list of indices and offsets and return a vector of averaged embeddings. One could rewrite this module using the previously defined operator:
def EmbeddingBag(emb, input, offsets, mode='mean'):
embedded_input = emb(input.unsqueeze(0))
return my_operator(embedded_input, offsets, mode=mode)
input = Variable(torch.LongTensor([1,2,4,5,4,3,2,9]))
offsets = Variable(torch.LongTensor([0,4]))
emb = nn.Embedding(10, 3)
EmbeddingBag(emb, input, offsets)
Is there a way to get this operator? Is it available in pytorch ? in torch ? I feel it could be very useful in many situations.