@cpeters, Most likely. It’s getting interesting.
I have executed the project, which I used to test AMP in Kaggle kernels, in my workstation. AMP seems to be working there. So yeah, seems like it’s some issues on Lighting side which my combination of model/dataset discovered 
In any case, thank you for the help!