Save and load model issue

1

Hello, dears :slight_smile:

I’m trying to implement the code of The Annotated Transformer from Harvard NLP, everything is working well but I need to save and load my own mode instead of their (already) trained one, which is iwslt.pt.

model_opt = NoamOpt(model.src_embed[0].d_model, 1, 2000,
            torch.optim.Adam(model.parameters(), lr=0, betas=(0.9, 0.98), eps=1e-9))

N_EPOCHS = 2
CLIP = 1

best_valid_loss = float('inf')

for epoch in range(N_EPOCHS):
    model_par.train()
    run_epoch((rebatch(pad_idx, b) for b in train_iter), 
                  model_par, 
                  MultiGPULossCompute(model.generator, criterion, 
                                      devices=devices, opt=model_opt))
    model_par.eval()
    valid_loss = run_epoch((rebatch(pad_idx, b) for b in valid_iter), 
                          model_par, 
                          MultiGPULossCompute(model.generator, criterion, 
                          devices=devices, opt=None))
        if valid_loss < best_valid_loss:
        best_valid_loss = valid_loss
        torch.save(model_par.state_dict(), 'model.pt')
        
    print(valid_loss)

For loading model I use :

model = torch.load("model.pt")

I got an error of
AttributeError: 'collections.OrderedDict' object has no attribute 'encode'

Also, I used :
model.load_state_dict(torch.load('en-de-model.pt'))

I got an error : AttributeError: 'collections.OrderedDict' object has no attribute 'load_state_dict'

Could you help to find the optimal way to save and load my won model?

2

After saving the model state dict like this: torch.save(model_par.state_dict(), 'model.pt')
In general, you need to do these steps:

  1. Initialize the new instance of the model the same way you initialize it before training and saving, like: model_par= MyFancyModelClass(some_args_here)
  2. Load state dict into model: model_par.load_state_dict(torch.load('model.pt'))