Hello,
I am comparing the same workflow accross InfiniBand (IB) and TCP. the workflow is one of the first example from the torch tune repo.
IB compute time 37.8min NCCL_IB_DISABLE=0
TCP compute time 32min NCCL_IB_DISABLE=1
Should IB be faster than TCP?
If needed I can add the yaml config