Loading data taking a lot of time

This might create a data loading bottleneck.
You could use the data_time meter from the ImageNet example to track the data loading time.

This post gives you some more information in case you haven’t read it already.