Error while trying to run run_wwm_mlm.py using my saved model: TypeError: ‘NoneType’ object is not iterable

nlp
1

I have trained a BertForSequenceClassification model, saved the model and tokenizer by

model.save_pretrained('output_mlm_cls')
tokenizer.save_pretrained('output_mlm_cls')

I’m trying to run run_mlm_wwm.py and I want it to load the saved model above:

python run_mlm_wwm.py \
    --model_name_or_path /path/to/output_mlm_cls \
    --train_file /path/to/my_data.txt \
    --do_train \
    --output_dir /output_dir

Then I get this error message:

Traceback (most recent call last):
File “run_mlm_wwm.py”, line 408, in
main()
File “run_mlm_wwm.py”, line 367, in main
train_result = trainer.train(resume_from_checkpoint=checkpoint)
File “/home/cl/jungmin-c/.pyenv/versions/anaconda3-5.1.0/envs/jp/lib/python3.7/site-packages/transformers/trainer.py”, line 1066, in train
self._load_state_dict_in_model(state_dict)
File “/home/cl/jungmin-c/.pyenv/versions/anaconda3-5.1.0/envs/jp/lib/python3.7/site-packages/transformers/trainer.py”, line 1387, in _load_state_dict_in_model
if set(load_result.missing_keys) == set(self.model._keys_to_ignore_on_save):
TypeError: ‘NoneType’ object is not iterable

I could not figure out what is wrong with my approach. I would greatly appreciate your suggestion.
Thank you!

2

Based on the error message it seems that one of the inputs to set is a None object as seen here:

set(None)
> TypeError: 'NoneType' object is not iterable

My guess would be that the transformers.trainer object is unable to load the desired file and returns None instead.
CC @stas for viz.

1 Like
3

Thanks for the ping, @ptrblck!

I’m not familiar with: transformers/run_mlm_wwm.py at master · huggingface/transformers · GitHub as it’s some experimental example I have never used and it’s not being maintained, so it’s very likely to be out of sync with the current code base. Therefore @jungminc88 please file an Issue at Issues · huggingface/transformers · GitHub

You probably need to tag the person who wrote this example, see the original PR:

1 Like