# Error while Multiprocessing in Dataloader **URL:** \ **Category:** Uncategorized\ **Created:** [June 2, 2019, 2:47am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845 "2019-06-02T02:47:42Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![gaurav0651](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gaurav0651/32/12579_2.png) [@gaurav0651](https://discuss.pytorch.org/u/gaurav0651)\ **Post date:** [June 2, 2019, 2:47am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/1 "2019-06-02T02:47:42Z")
Not sure if this is reported already but I am getting the following Assertion error in Dataloader Exception ignored in: \\> Traceback (most recent call last): File “/home/amit/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 677, in **del** self.\_shutdown\_workers() File “/home/amit/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 659, in \_shutdown\_workers w.join() File “/usr/lib/python3.6/multiprocessing/process.py”, line 122, in join assert self.\_parent\_pid == os.getpid(), ‘can only join a child process’ AssertionError: can only join a child process ---
**Author:** ![matohak](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/df788c/32.png) [@matohak](https://discuss.pytorch.org/u/matohak)\ **Post date:** [January 24, 2020, 2:42pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/2 "2020-01-24T14:42:48Z")
have you found a solution? ---
**Author:** ![uduse](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/uduse/32/21342_2.png) [@uduse](https://discuss.pytorch.org/u/uduse)\ **Post date:** [March 8, 2020, 6:01am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/3 "2020-03-08T06:01:19Z")
I am using `num_workers` with `IterableDataset` and it also has this problem. ---
**Author:** ![karzia](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/karzia/32/15829_2.png) [@karzia](https://discuss.pytorch.org/u/karzia)\ **Post date:** [May 8, 2020, 1:25am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/4 "2020-05-08T01:25:30Z")
yes… same here. testset = torch.utils.data.DataLoader(trainset, batch\_size=4, shuffle=True,num\_workers=0) But i just worried that it is possible to use only my local testing. So I just want to know what is root cause and a solution. ^^ ---
**Author:** ![sharkdeng](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sharkdeng/32/23021_2.png) [@sharkdeng](https://discuss.pytorch.org/u/sharkdeng)\ **Post date:** [July 22, 2020, 2:28am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/5 "2020-07-22T02:28:24Z")
> [@gaurav0651](#): > > can only join a child process did you solve it. Or it is the problem of num\_workers. ---
**Author:** ![Himani\_Gulati](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/himani_gulati/32/25526_2.png) [@Himani\_Gulati](https://discuss.pytorch.org/u/Himani_Gulati)\ **Post date:** [August 13, 2020, 8:48pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/6 "2020-08-13T20:48:59Z")
Well I am getting the same error, it says can only join a child process. I do not know what that means?? ---
**Author:** ![Gabriel\_Barello](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/gabriel_barello/32/27983_2.png) [@Gabriel\_Barello](https://discuss.pytorch.org/u/Gabriel_Barello)\ **Post date:** [August 20, 2020, 5:17am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/7 "2020-08-20T05:17:34Z")
I was having this issue. Turns out its because there was an error in the dataset object (for me it was in the ` __getitem__ ` function). I guess the DataLoader in multiprocessing mode doesn’t know how to cleanly provide you with the internal error message. If you have the same problem, try running with `num_workers = 0` (single-threaded) and it should tell you what the error is. Once you’ve fixed the error, it should work with `num_workers > 0`. ---
**Author:** ![Qinsheng\_Zhang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/qinsheng_zhang/32/17929_2.png) [@Qinsheng\_Zhang](https://discuss.pytorch.org/u/Qinsheng_Zhang)\ **Post date:** [October 1, 2020, 4:44am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/8 "2020-10-01T04:44:10Z")
same issue happens. Not know where is the error. ---
**Author:** ![jpleet](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jpleet/32/30125_2.png) [@jpleet](https://discuss.pytorch.org/u/jpleet)\ **Post date:** [October 21, 2020, 8:44pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/9 "2020-10-21T20:44:35Z")
I just got this error. My data is good. And training with _num\_workers=0_ is too slow. For whatever reason, I was able to fix by replacing _from tqdm.auto import tqdm_ with just _from tqdm import tqdm_ Something seems to bug out with parallel dataloaders wrapped around the fancy notebook tqdm with my versions of nodejs and ipywidgets. Hope this helps others. ---
**Author:** ![JamesLai](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jameslai/32/31738_2.png) [@JamesLai](https://discuss.pytorch.org/u/JamesLai)\ **Post date:** [November 30, 2020, 2:42pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/11 "2020-11-30T14:42:29Z")
This solve my problem. Thanks a lot! ---
**Author:** ![Hmrishav\_Bandyopadhy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hmrishav_bandyopadhy/32/24065_2.png) [@Hmrishav\_Bandyopadhy](https://discuss.pytorch.org/u/Hmrishav_Bandyopadhy)\ **Post date:** [December 12, 2020, 8:59am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/12 "2020-12-12T08:59:46Z")
This is literally gold! ---
**Author:** ![brianw0924](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brianw0924/32/36321_2.png) [@brianw0924](https://discuss.pytorch.org/u/brianw0924)\ **Post date:** [March 26, 2021, 2:03pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/13 "2021-03-26T14:03:04Z")
This solves my problem, too. but why??? ---
**Author:** ![adrofa](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adrofa/32/30459_2.png) [@adrofa](https://discuss.pytorch.org/u/adrofa)\ **Post date:** [April 1, 2021, 10:08am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/14 "2021-04-01T10:08:37Z")
This helped! Thank you! ---
**Author:** ![DarQ](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/darq/32/39606_2.png) [@DarQ](https://discuss.pytorch.org/u/DarQ)\ **Post date:** [June 23, 2021, 8:21pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/15 "2021-06-23T20:21:41Z")
I’m not even using tqdm and my code works fine with num\_workers=0. What could be the problem? ---
**Author:** ![akos.kiss](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/e95f7d/32.png) [@akos.kiss](https://discuss.pytorch.org/u/akos.kiss)\ **Post date:** [June 27, 2021, 6:23am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/16 "2021-06-27T06:23:23Z")
Same here, no tqdm, code worked with num\_workers=0, 1, 2, but saw a lot of these errors when num\_worers\>=3. I ran the code inside docker and increasing the shared memory size (–shm-size 256M → 1G) solved the problem for me, now works fine with num\_workers=12. ---
**Author:** ![Fang\_Nan](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fang_nan/32/41899_2.png) [@Fang\_Nan](https://discuss.pytorch.org/u/Fang_Nan)\ **Post date:** [September 3, 2021, 2:27pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/17 "2021-09-03T14:27:09Z")
That works for me. Thanks! ---
**Author:** ![drscotthawley](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/drscotthawley/32/118_2.png) [@drscotthawley](https://discuss.pytorch.org/u/drscotthawley)\ **Post date:** [November 2, 2021, 3:56am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/18 "2021-11-02T03:56:49Z")
Not using tqdm but changing `num_workers` from 1 to 0 caused this error to go away on my Colab run! 🙂 ---
**Author:** ![Noyii](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/noyii/32/45096_2.png) [@Noyii](https://discuss.pytorch.org/u/Noyii)\ **Post date:** [December 20, 2021, 12:37pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/19 "2021-12-20T12:37:33Z")
that really helps by replacing _from tqdm.auto import tqdm_ with just _from tqdm import tqdm_ Thx a lot! ---
**Author:** ![steve\_ari](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/steve_ari/32/45703_2.png) [@steve\_ari](https://discuss.pytorch.org/u/steve_ari)\ **Post date:** [May 3, 2022, 7:31am UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/20 "2022-05-03T07:31:32Z")
I just changed the `num_workers` to `num_workers=0`, run the training once and then change it back again to `num_workers=4`. The error just disappeared afterwards ---
**Author:** ![agg-shambhavi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/agg-shambhavi/32/38333_2.png) [@agg-shambhavi](https://discuss.pytorch.org/u/agg-shambhavi)\ **Post date:** [May 29, 2022, 1:26pm UTC](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845/21 "2022-05-29T13:26:41Z")
> [@jpleet](#): > > from tqdm import tqdm The warnings were annoying me a lot. Thanks :)) [Next page](https://discuss.pytorch.org/t/error-while-multiprocessing-in-dataloader/46845.md?page=2)