How to make a loss function for nn.Transformer

1

I’m working on building a transformer that translates polynomials into specific sequences of numbers. However, I’m running into a lot of trouble with the loss function.

I understand that when you use nnTransformer, the source input needs to be size (S,N,E) (where S is the length of the input) and the corresponding target needs to be (T,N,E) (where T is the target length).

The transformer returns an output of size (T,N,E).

I tried to implement nn.CrossEntropyLoss to calculate the loss, but when I do I get this error:

IndexError: Target -1 is out of bounds.

I don’t understand what is causing this error. Could anyone help me understand what it means so I can fix the problem? The tensors that I am sending to nn.CrossEntropyLoss are sizes [25, 32, 512] and [25, 512].

What could be causing this error? Any help is appreciated! Thanks!

2
  1. Make sure that the target vocab is created correctly. The target labels for a batch are as expected. Also, if you’ve created your custom loss function that might have an issue.
  2. Is possible share your code.