Gpu waits for cpu so long time

The first several batchsize training speech is ok, the utilization rate of cpu is ok, after several batchsize, cpu becomes slower, gpu waits for cpu so long time? Please give me some suggestions. Thanks very much.

Could you explain what “cpu becomes slower” means? Do you see a decrease in its frequency or any other performance limiting issue? If so, could you check, if your system might be overheating?