How to write parallel for loop in model's forward method?

2

I think you could use torch.distributed.launch. This snippet may be useful to you to get a better grasp on how torch.distributed.launch works.

1 Like