recently, i’ve been seeing warnings saying that you need to add a ‘dim’ argument to Softmax as the implicit dimension selection is being deprecated.
I have a model that I found on github that uses a softmax layer (nn.LogSoftmax) in its forward function and an F.softmax() in its inference functions.
The data i’m feeding in has dimensions batch_size x output_classes. So, I replaced the call to F.softmax with F.softmax(___, dim=2)
This results in the very confusing and nonsensical error:
RuntimeError: Dimension out of range (expected to be in range of [-2, 1], but got 2)
I don’t … get why I would ever try to take a softmax over a negative dimension. Additionally, I don’t understand why this is stopping me from taking a softmax over the second dimension, when that is actually the dimension that I want to take the softmax over.
I tried swapping out the call to F.softmax with a call to self.softmax, which refers to the nn.LogSoftmax layer initialized in the model’s init() function with dim=2 as an argument, and got the same error, despite this softmax working during training.
I don’t understand what these errors mean, they seem totally illogical, and I don’t understand what the difference is between an nn.LogSoftmax and F.softmax in the first place.