Why multiprocess behavior differ between v1.12 and v1.9

When I use torch==1.9.0, the following code runs fine.

import torch
from multiprocessing import Process
import multiprocessing

def run():
    print('in proc', torch.cuda.is_initialized())
    print('in proc', torch.cuda._is_in_bad_fork())

def fk_run():
    print('in fork proc', torch.cuda.is_initialized())
    print('in fork proc', torch.cuda._is_in_bad_fork())

if __name__ == "__main__":
    print('in main', torch.cuda.is_initialized())
    print('in main', torch.cuda._is_in_bad_fork())

    sp_ctx = multiprocessing.get_context('spawn')
    a = sp_ctx.Process(target=run)

    fk_ctx = multiprocessing.get_context('fork')
    b = fk_ctx.Process(target=fk_run)

However, when run with torch=1.12 get a RuntimeError

RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

I check the source code of 1.9 and 1.12 and find no difference.

Besides, the demo code doesn’t init CUDA in the main process, why prompts Cannot re-initialize CUDA in forked subprocess ?

In PyTorch 2.0.0 I’m seeing:

in main False
in main False
in fork proc False
in fork proc False
in proc False
in proc False

Ok. Still confused by the v1.12.0’s RuntimeError.

I don’t know what might have caused this issue in this older version, but are you still seeing the same issue in 2.0.0 or a recent nightly?

No,2.0.0 works fine.

OK, great! Thanks for confirming.