Hi
I am working on an environment in Reinforcement Learning based which is based on Graph network. I am trying to implement the Actor-Critic network. Here I have two agents which are following the policy. I encounter this error (code is shown below with error) most of the time when I run the model for a certain amount of episodes with specific learning rates. Very few times the program executed successfully. But lot of times it throws an error stating ‘invalid multinomial distribution’. I am using Categorical Distribution and I dont think I have any values with probabilities less than zero.
I am pasting the line where I am getting the error.
I have gone through the answer (https://discuss.pytorch.org/t/categorical-probs-sample-generates-runtimeerror-invalid-argument-2-invalid-multinomial-distribution-encountering-probability-entry-0/27386) which was given for the same kind of error, but I could not figure out any solution.
def getAction1(self, state):
state = torch.FloatTensor(state)
logits, _ = self.model1.forward(state)
dist = F.softmax(logits, dim = -1)
probs = Categorical(dist)
return probs.sample()
The error
File "C:/Users/Prudhvinath.DESKTOP-09Q8801/sciebo/Thesis/JSSP/TwoAgents/20JobsTwoAgents.py", line 164, in getAction1
return probs.sample()
File "C:\Users\Prudhvinath.DESKTOP-09Q8801\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample
sample_2d = torch.multinomial(probs_2d, 1, True)
RuntimeError: invalid multinomial distribution (encountering probability entry < 0)
The Line164
redirects to this line in the main code return probs.sample()
Can you please help me here?
thanks in advance.