Face recognize with Depthmap

Hi everybody.
Sorry my English not good. Im from Viet Nam
I have question, My Project about recognize identify. I using camera Intel D435 to collect and get depthmap of face, I folllow this paper with my dataset student. link
After I train Model include one pictures(.png) and one depthmap(.npy) of face, I train my model ResNet50 with pretrain Imagenet, using Pytorch, my dataset have 500 class with 70 picture/class. I got model 99.7% But i have problem my model only veryfi data trained in my dataset, and not verify real data.
My data with ratio 8:1:1 8 train, 1 eval, 1 test
I think my model is overfit or something wrong??. Please help me.
How i can fine tune or do something to improve accuracy in real data.??

Link paper: GitHub - XingwXiong/Face3D-Pytorch: 3D Face Recognition, implemented with PyTorch