nn.Embedding with a dynamic number of embeddings?

1

Hi, I am wondering whether you already saw an implementation of nn.Embedding for which the number of embeddings could be dynamic, with the following behaviour:

if you give it an embedding index that is not in the current set of indexes for embeddings, allocate a new embedding for that.

if you would have an elegant way to do that in mind ? I guess it’s not trivial since the optimizer’s parameters would need to change, etc etc

thanks

antoine

1 Like