I have access to HPC node, The maximum wall time for the GPU node I have access to is 12 hours. I have access to a maximum of 4 Tesla V100-PCIE-16GB. And about 36 CPU cores. I wondered if DDP could allow me to use all the cores alongside GPUs. I would love a link to a tutorial resource that is possible. Thanks.
Probably looking something similar to this https://researchcomputing.princeton.edu/support/knowledge-base/pytorch