I’m trying to evaluate a pretrained model that i trained with the PPO example from the pytorch tutorials website but i’m failing to obtain the same results as during training.
I’ve saved the model (in the train script) with
model_weights_filename = "models/ppo_example_model_weights_500k.pth"
torch.save(policy_module.state_dict(), model_weights_filename)
and loaded them with
model_weights_filename = "models/ppo_example_model_weights_500k.pth"
policy_module.load_state_dict(torch.load(model_weights_filename))
in my evaluation script.
Finally i’m using the same rollout function as in the tutorial (the policy_module is defined equal to the train script):
""" Evaluate """
torch.manual_seed(37)
policy_module.eval()
with torch.no_grad():
for _ in range(10):
eval_rollout = env.rollout(1000, policy_module)
In some rare cases i’m getting good results, but most of the time the actor survives only a few steps instead of 999.
Am I missing something crucial here?
Thanks in advance
Juri