Hi, I am wondering whether you already saw an implementation of
nn.Embedding for which the number of embeddings could be dynamic, with the following behaviour:
if you give it an embedding index that is not in the current set of indexes for embeddings, allocate a new embedding for that.
if you would have an elegant way to do that in mind ? I guess it’s not trivial since the optimizer’s parameters would need to change, etc etc