No speed increase when using multiple GPUs

Increasing the global batch size might need changes in your training hyperparameters. I think this topic discusses some differences in detail, so you could check if some of them are also valid for your use case.