Hi everyone, when I train the model with 4 GPUs using pytorch distributed, I see error messages 4 times if there is any, how to make it only appear once? Thanks
If you start the training using a function that gets a
rank argument passed, I used this:
if rank == 0: print("error message")
If you want to suppress both stdout and stderr outputs, you can try this:
import sys, os f = open(os.devnull, "w") if rank != 0: sys.stdout = f sys.stderr = f
Thanks. It is exactly what I wanted.