Ignore_index in Adaptive Softmax?

I am training a large scale neural language model with pytorch and would like to use an Adaptive Softmax (because my vocabulary is very large) function over the outputs, this is provided in pytorch by torch.nn.AdaptiveSoftMaxWithLoss. This function computes the loss for me as well as the adaptive softmax. However, my input data is padded to a certain length with a padding token. Normally when I use CrossEntropyLoss or NLLLoss there is the parameter ignore_index which allows me to ignore the padding token when computing the loss. It does not seem like such a parameter is available for the adaptive softmax implementation, how do I work around this?