I want to print error messages only once when using multi-GPU

distributed
1

Hi everyone, when I train the model with 4 GPUs using pytorch distributed, I see error messages 4 times if there is any, how to make it only appear once? Thanks

2

If you start the training using a function that gets a rank argument passed, I used this:

if rank == 0:
    print("error message")
1 Like
3

If you want to suppress both stdout and stderr outputs, you can try this:

import sys, os

f = open(os.devnull, "w")
if rank != 0:
    sys.stdout = f
    sys.stderr = f
2 Likes
4

Thanks. It is exactly what I wanted.