Dataloader, multiple workers, and KeyboardInterrupt

Often it’s nice to design your code to kill cleanly without traceback with a KeyboardInterrupt. However, it seems like when using multiple workers and the data loader, KeyboardInterrupt doesn’t get caught correctly with a wrapping try/except (this is a known problem with multiprocessing). Is there any work-around for this?


1 Like


The patch below doesn’t seem to handle KeyboardInterrupts. Nevermind my old post.

Old post:

There’s a timeout option for DataLoader on the master branch, but you’ll have to install master from source until PyTorch updates the binaries.

I’ve figured out a partial hack for this which works slightly better, but not perfectly. When initializing the DataLoader:

def worker_init(x):
    signal.signal(signal.SIGINT, signal.SIG_IGN)

loader = DataLoader(dataset, batch_size=batch_size, shuffle=shuffle, num_workers=n_workers,

Instead of exploding the terminal, I get:

Exception ignored in: <bound method _DataLoaderIter.__del__ of < object at 0x7fcb5c36c358>>
Traceback (most recent call last):
TypeError: 'NoneType' object is not callable

So: better, but not perfect.


Thanks for this wonderful workaround! Worked like a charm. I didn’t even get the Exception ignored message. :smiley:

The Exception ignored is probably due to that I’m nesting some KeyboardInterrupt exception catching.

I wouldn’t be surprised if there are some unintended side-effects to this, but so far it seems to work well. I’m worried though that the workers might not be terminating correctly. Perhaps someone more familiar with multiprocessing can help.