Reference for scale_grad_by_freq option in nn.Embedding

nlp
1

I found the scale_grad_by_freq option in the nn.Embedding module.

However, I couldn’t find the reference for that option.

Does anyone know which survey or paper used this technique?

Thank you.

1 Like
2

Can anyone provide some updates on this? I wanted to know the motivations and the results of using this technique for learning embeddings…