How to disable these two types of log output?

Hello,I got two types of log output:

First type which comes from Dataloader:

INFO     process shutting down 2024-12-21 20:51:22,045
INFO     process shutting down 2024-12-21 20:51:22,045
INFO     process exiting with exitcode 0 2024-12-21 20:51:22,045
INFO     process exiting with exitcode 0 2024-12-21 20:51:22,046
INFO     process shutting down 2024-12-21 20:51:22,046
INFO     process exiting with exitcode 0 2024-12-21 20:51:22,046
INFO     process shutting down 2024-12-21 20:51:22,046
INFO     process shutting down 2024-12-21 20:51:22,046

Second type which comes from torch.distributed

sc:14361:15083 [0] NCCL INFO AllReduce: opCount 0 sendbuff 0x7af275e06400 recvbuff 0x7af275e06400 count 3921 datatype 7 op 0 root 0 comm 0x14e99380 [nranks=1] stream 0x13d72800

I have tried to set log level like this:

torch._logging.set_logs(all = logging.WARN)

but it doesn’t work.

Question is how to get rid of these two types log?

The NCCL log is printed since you’ve exported the NCCL_DEBUG env variable. unset it and the logs will disappear.

Thanks,NCCL log gone by following your suggestion,
would you please give some suggestions about that Data

Loader log?
INFO     process shutting down 2024-12-21 20:51:22,045
INFO     process shutting down 2024-12-21 20:51:22,045
INFO     process exiting with exitcode 0 2024-12-21 20:51:22,045
INFO     process exiting with exitcode 0 2024-12-21 20:51:22,046
INFO     process shutting down 2024-12-21 20:51:22,046
INFO     process exiting with exitcode 0 2024-12-21 20:51:22,046
INFO     process shutting down 2024-12-21 20:51:22,046
INFO     process shutting down 2024-12-21 20:51:22,046

I don’t kow where these logs come from as I haven’t seen them in PyTorch. A quick forum search suggests you might have enabled them via logger = multiprocessing.log_to_stderr().

Thanks,these logs comes from the workers of DataLoader

DataLoader(trainDataSet, batch_size = len(x_train), num_workers = 1)

set
num_workers = 1
will print out one line, set
num_workers = n
will print out n lines