DDPG Algorithm with Libtorch is not able to learn


#1

Hey there,
I implemented a DDPG algorithm in libtorch (v. 1.0.1), but training was never successful. It seems as if the network is never updated.
Before that, I did the same in PyTorch and it worked like a charm. I couldn’t figure out what I did wrong in c++.

Here’s my code:

For testing the implementation I used the OpenAiGym Pendulum-v0 framework.
(with c++ api: https://github.com/openai/gym-http-api)

It would be great if someone could help me out
-Thank you-


#2

Found out, that I messed up the soft_update function:
it has to be:

torch::NogradGuard no_grad;
for (size_t i=0; i < target->parameters().size(); i++) {
    target->parameters()[i].copy_(tau*local->parameters()[i] + (1.0 - tau) * target->parameters()[i]);
}