Pytorch distributed with gloo backend send float32 or float64

distributed
1

Hello,
I am implementing MPI and compare to pytorch distributed,
it’s seems that there is mismatch between my implemetation and the one of pytorch with gloo.

does pytorch distributed send float32 or float64 tensors?

2

It depends on what the scalar type of the tensor you passed to the communication API. Check out this.