Error loading state_dict ; GPT2 Finetuning

I was using this snippet of code to load my finetuned GPT2 and it was working absolutely fine:
tokenizer = GPT2Tokenizer.from_pretrained(‘gpt2-medium’)
model = GPT2LMHeadModel.from_pretrained(‘gpt2-medium’)
model_path = ‘/content/drive/My Drive/PREDICTION/POETRY/EDset3_trained_models/’
model =

As of this morning, having made no changes, I get the following error:
RuntimeError: Error(s) in loading state_dict for GPT2LMHeadModel:
Missing key(s) in state_dict: “transformer.h.0.attn.masked_bias”, “transformer.h.1.attn.masked_bias”

Hi Kiran,

is this issue solved here already or are you still running into these issues?

1 Like