I have two nn.Embeddings, A and B, with different embeddings num. B contains A. For an example:
index weight 1 [0.1, 0.2] 2 [0.4, 0.6]
index weight 1 zeros 2 zeros 3 zeros
Now I want to copy the weights in A to B. After copy B should be like:
index weight 1 [0.1, 0.2] 2 [0.4, 0.6] 3 zeros
Any workable method? I only know a function from_pretrained can achieve weight loading, but it requires the same structure.
Or… anyway to expand A? I don’t know if nn.Embedding is static.