Multiprocessing cause model's parameters all become to 0.0


In order to improve the sampling efficiency I decide to use multiprocessing to do the sampling with CPU(model prediction also with cpu) and train the model with single a process by GPU.

The multiple rollout workers code as followed:

def multi_sampling(_env):
    #'s and _env.critic's named_parameters() became all 0 **after training**. 
    # The parameters are correct before training

    agent = Agent(_env)
    sample_res = agent.sampling()
    return sample_res

env = CartPole()       # _env has my actor model and critic model
process_pool = Pool(env.n_WORKER)
for sample_index in range(env.n_SAMPLES_PER_EPISODE):
    res = process_pool.apply_async(multi_sampling, args=(env,))   
    res_dict[sample_index] = res

After training the models(update the weights of actor and critic), the model’s weights and biases becomes 0.0 in function multi_sampling. But the weights and biases are correct before my first training.

If I only use CPU to do the sampling and training, the results will just perfect which I could finish the cartpole task in 10 seconds (avg 195 rewards of continuous 100 episodes)

My question is, how do I sampling with CPU and training with GPU in a efficient and correct way?

use cpu to sample and train will be fine.

@ptrblck Hi Ptrblck, could you please help me with this problem? Or could you tell me what should I do if I want to sample on CPU and train on GPU.

Sorry I don’t fully understand the use case and e.g. don’t know what the to() operation in “Update Weights” refers to.
The usual use case would be to use a DataLoader with multiple workers for the data loading and push the data to the GPU once it’s ready.
Based on your issue I guess you might be running into multiprocessing issues e.g. as described here.

Thanks for the response, I will try dataloader.