# Bizarre InternalTorchDynamoError with locally and formerly working code

**URL:** <https://discuss.pytorch.org/t/bizarre-internaltorchdynamoerror-with-locally-and-formerly-working-code/216382>\
**Category:** Uncategorized\
**Created:** [February 7, 2025, 9:26pm UTC](https://discuss.pytorch.org/t/bizarre-internaltorchdynamoerror-with-locally-and-formerly-working-code/216382 "2025-02-07T21:26:52Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [February 7, 2025, 10:51pm UTC](https://discuss.pytorch.org/t/bizarre-internaltorchdynamoerror-with-locally-and-formerly-working-code/216382/5 "2025-02-07T22:51:29Z")

</div>

To launch `torchrun` on multiple devices you would use `torchrun --nproc_per_node==8 ...` which will then correspond to the `--local-rank` argument inside your script as described [here](https://github.com/pytorch/pytorch/blob/91dfa829811dacadf77ecf082ae4cd5957ff2508/torch/distributed/run.py#L44-L61).  
In your approach you are launching your script with `torchrun` only and are not using the `--local-rank` at all, so again unsure how this should have ever worked.  
Alternatively, you can also use a multiprocessing approach inside your script which will spawn the processes there as described in [this tutorial](https://pytorch.org/tutorials/intermediate/dist_tuto.html).

---

_[View the full topic](https://discuss.pytorch.org/t/bizarre-internaltorchdynamoerror-with-locally-and-formerly-working-code/216382)._