Two questions, first is that I have a network that I would like to train for a little while using dataparallel and then modify the network and continue to train with dataparallel. What is the correct way to reinitialize? Should I remove the dataparallel in some way or just call it again on net.module(which runs but doesn’t appear to work as expected)?
Second question is continued from here: How to transition to functions not being allowed to have member variables. Now that I am using a dictionary it appears the tensors in that dictionary are not split across GPUs. What’s the best way to include them in the dataparallel?