I’m trying to learn how to load pretrained glove vectors using torchtext and I manage to get something to work but I’m confused of what it’s doing.
As an example I have something like this:
Then in my model I have 100 dimensional embeddings and I load these weights by
pretrained_embeddings = MyField.vocab.vectors my_model.embedding.weight.data.copy_(pretrained_embeddings)
So it’s loading the pretrained embedding matrix which has been trained on a vocabulary of some size, how does that work with a vocabulary that is of different size and different words? My idea is that it should copy the vectors for the words that is in my vocabulary, and randomly initialize the rest, is that what it’s doing?