Is there will have total 48g memory if I use nvlink to connect two 3090?

distributed
10

Yes, you should be able to use nn.DataParallel with any GPUs and with your 3060s also.
As @spacecraft1013 explained you won’t necessarily need nvlink, but it would speed up the p2p communication.

Also note, that we generally recommend to use DistributedDataParallel with a single process per device for the best performance.

2 Likes