How to fix finetuned model?

I finetuned gpt-j model by fp16.

After the trainning, the model makes abnormal memory usage.

Long tokens make much memory use.

Short tokens need double memory usage too.

So I extracted the files before and after, But they aren’t have big difference.

it was little volume, few MB.

I think it needs some converting, or edit options for light model.

How can I find cause of problems?