Torch Distributed Class Definitions

Vibhatha_Abeykoon · February 27, 2020, 9:38pm

Hi,

I am trying to locate ReduceOp definitions, referred in the following comment in the source.

pytorch/pytorch/blob/243af17d65fb147901bf9efef21b0cb3e4f25fee/torch/distributed/distributed_c10d.py#L92


# TODO: remove them when users are ready to take a hard dependency on PyTorch 1.
_backend = Backend.UNDEFINED
dist_backend = Backend




class reduce_op(object):
    r"""
    Deprecated enum-like class for reduction operations: ``SUM``, ``PRODUCT``,
    ``MIN``, and ``MAX``.


    :class:`~torch.distributed.ReduceOp` is recommended to use instead.
    """


    def __init__(self):
        # __members__ is a dict storing key-value pairs for enum classes
        for k, v in ReduceOp.__members__.items():
            setattr(self, k, v)
        self.__members__ = ReduceOp.__members__


    def __getattribute__(self, key):
        warnings.warn("torch.distributed.reduce_op is deprecated, please use "

(I did search the git source and local installation both)

I couldn’t find the class definition. Am I missing something here?

osalpekar · February 27, 2020, 10:18pm

ReduceOp is a C++ enum, and is exposed to the python interface using pybind (https://github.com/pytorch/pytorch/blob/master/torch/csrc/distributed/c10d/init.cpp#L145). That enum is defined here: https://github.com/pytorch/pytorch/blob/master/torch/lib/c10d/Types.hpp#L8