Training with data parallel , validation on 1 gpu

1

Hi, I’m using hooks for validation, and with data parallel only the packet of gpu=0 returns, I saw that there is not advance with that issue.
So I thought on a hack, I will train with data-parallel but when doing validation I’ll remove the data parallel, and send the model to 1 gpu only.
No matter what I tried to do it did not succeed, “to(‘cuda:0’)”, sending it to cpu, no matter what it returns half packet.
So now to my question, how can I take a model with data-parallel wrapper, and remove the data-parallel wrapper of him?!!?

2

After you’ve wrapped the model into nn.DataParallel you can get the non-parallel model back via:

model = model.module

Let me know, if you are still seeing any issues.