torch::nn::DistributedDataParallel C++ class?

distributed
1

Hi everyone, I was wondering: why isn’t the DistributedDataParallel a C++ core class, while it seems to be a Python only feature?

2

I believe DistributedDataParallel theoretically can be exposed as part of C++ frontend. We are open to contribution PR if you are interested in contributing.

3

I understand. Can you just confirm that as per the current status, DDP is not available from the C++ frontend?
I can definitely consider contributing, though for the moment the simplest solution was to upstream my changes up to the Python frontend.

4

Correct, there is no C++ frontend for DDP. DDP mainly leverages reducer (pytorch/torch/csrc/distributed/c10d/reducer.cpp at main · pytorch/pytorch · GitHub) which is written in C++, but we have no explicit API for DDP at the C++ level.