How to create a great training loop?


I am more advanced in Deep Learning, but I wanted to develop further and see what can be improved.
I am currently looking for some resources to improve the training process (= the loop where the backpropagation takes place).
This does not automatically have to mean better performance etc.
One example I can think of is to use Tensorboard for logging. Do you have any other ideas or a good resource?

Thanks for the help