RuntimeError in PyTorch's reductions.py when using multiprocessing.Queues with multiple threads

2

I think your explanation is correct and some thread/process manipulates the dict, while the loop is still being executed. Here is a minimal code snippet to reproduce this underlying issue:

x = {a: a+10 for a in range(10)}
for key in x:
    print(key, x[key]) # works

for key in x:
    print(key, x[key])
    x[key+100] = key
> RuntimeError: dictionary changed size during iteration

Unfortunately, I don’t know enough about your setup and which process could potentially change the self.items() dict. In any case, you could add debug statements to see when the self.items() objects is being manipulated, which might help in isolating the issue.