On a multi core system, where we are using MPI to run parallel threads of our pytorch program.
if we run:
model.to(device=torch.device(F"cpu:{device_id}"))
Does the “device_id” correspond to the “rank” of the MPI thread? Or should we send everything to cpu:0?