What is torch.cuda.empty_cache do and where should i add it?

[ptrblck]

It’s not, as it will synchronize your code and free all cached memory. Since no memory is in the cache, the next allocations will again synchronize your code during the cudaMalloc calls and thus cause slowdowns in your code.