# Parallel async inference of multiple models gives CUDA error: initialization error

**URL:** <https://discuss.pytorch.org/t/parallel-async-inference-of-multiple-models-gives-cuda-error-initialization-error/174897>\
**Category:** Uncategorized\
**Created:** [March 15, 2023, 8:24am UTC](https://discuss.pytorch.org/t/parallel-async-inference-of-multiple-models-gives-cuda-error-initialization-error/174897 "2023-03-15T08:24:45Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![zshn25](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zshn25/32/57310_2.png) [@zshn25](https://discuss.pytorch.org/u/zshn25)\
**Post date:** [March 15, 2023, 8:24am UTC](https://discuss.pytorch.org/t/parallel-async-inference-of-multiple-models-gives-cuda-error-initialization-error/174897/1 "2023-03-15T08:24:45Z")

</div>

I am trying to run inference of multiple models parallelly on CUDA. My use case is to run them on the same input image. I get the following error.

```auto
RuntimeError: CUDA error: initialization error
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

```

Even with `CUDA_LAUNCH_BLOCKING=1` I get `RuntimeError: CUDA error: initialization error`

A MWE would be

```python
import torch
from torch import nn
import torch.multiprocessing as mp
from torchvision import models

if __name__ == ' __main__':

    num_processes = 2

    input_image_pytorch = torch.randn((1,3, 256, 256), requires_grad=False, device="cuda")

    processes = []   
    for rank in range(num_processes):
        model = models.resnet18().eval().to("cuda")
        p = mp.Process(target=model, args=(input_image_pytorch,))
        p.start()
        processes.append(p)
    for p in processes:
        p.join()

```

---

<div class="post-metadata">

**Author:** ![zshn25](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zshn25/32/57310_2.png) [@zshn25](https://discuss.pytorch.org/u/zshn25)\
**Post date:** [March 15, 2023, 10:38am UTC](https://discuss.pytorch.org/t/parallel-async-inference-of-multiple-models-gives-cuda-error-initialization-error/174897/2 "2023-03-15T10:38:37Z")

</div>

Solved this by adding `mp.set_start_method('spawn')`