Hi,
I launched distributed training, and there is a tensor that is identical in each process, I would like this tensor to keep only one copy in the memory and can be accessed by multi-process.
However, when I defined the tensor and run share_memory_()
. It seems that all the processes still defines their own tensor and all of them moved their tensors into the shared memory which does not save memory at all.
Would you tell me how could I save memory in this way?