@ptrblck Many thanks for your help and reply. Sorry, do you have any idea about my post here?
Help CUDA error: out of memory - PyTorch Live - PyTorch Forums
The model can’t be fine tuned properly by using ddp and it generate nothing for me. I would appreciate if you can have a look. many thanks.