Unusual CPU Stalls and Significant Training Speed Unstable During First Epoch

2

And I also found this might be relative: CPU thread slow to enqueue GPU and communication kernels but i’m not sure.