I was interested in learning that, with deepspeed, you can use torch.distributed.run to run HuggingFace and Pytorch Lightning application with deepspeed optimizations. Is this also true for any arbitrary deepspeed application? Additionally, I was also curious as to what the differences were between the deepspeed launcher and using the torch.distribued.run launcher?
Thank you so much!