RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

Hi

I am working on an environment in Reinforcement Learning based which is based on Graph network. I am trying to implement the Actor-Critic network. Here I have two agents which are following the policy. I encounter this error (code is shown below with error) most of the time when I run the model for a certain amount of episodes with specific learning rates. Very few times the program executed successfully. But lot of times it throws an error stating ‘invalid multinomial distribution’. I am using Categorical Distribution and I dont think I have any values with probabilities less than zero.
I am pasting the line where I am getting the error.
I have gone through the answer (https://discuss.pytorch.org/t/categorical-probs-sample-generates-runtimeerror-invalid-argument-2-invalid-multinomial-distribution-encountering-probability-entry-0/27386) which was given for the same kind of error, but I could not figure out any solution.

def getAction1(self, state):
        state = torch.FloatTensor(state)
        logits, _ = self.model1.forward(state)
        dist = F.softmax(logits, dim = -1)
        probs = Categorical(dist)
        return probs.sample()

The error


  File "C:/Users/Prudhvinath.DESKTOP-09Q8801/sciebo/Thesis/JSSP/TwoAgents/20JobsTwoAgents.py", line 164, in getAction1
    return probs.sample()

  File "C:\Users\Prudhvinath.DESKTOP-09Q8801\Anaconda3\lib\site-packages\torch\distributions\categorical.py", line 107, in sample
    sample_2d = torch.multinomial(probs_2d, 1, True)

RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

The Line164 redirects to this line in the main code return probs.sample()

Can you please help me here?

thanks in advance.

Hello Prudvinath!

Your error message says that the error occurs “in getAction1,”
but you’ve posted code for def getAction2(self, state).

Could you double-check whether the code you’ve posted is
relevant to your error?

(Also, when your error gives a line number, “line 164,” it would
be helpful if you could flag the line in question when you post your
code so we can be sure of what we’re looking at.)

Best.

K. Frank

Hi Frank,

Thank you so much for the response. I have just now checked my question. Since there are two getAction i.e getAction1and getAction2 in my code and both being similar, I have posted one.

Now I have updated the question.

Thanks and regards
Prudhvi