I want to use multi task learning with ctc and forced teaching.

```
log_probs_ctc, log_probs_decoder = model(inputs.float(), targets_one_hot, device)
output_ctc_ = Variable(log_probs_ctc.transpose(1, 2).transpose(0, 1), requires_grad=True)
loss = ctc_criterion(output_ctc_, targets_ctc, input_lengths.type(torch.int32), target_lengths)
log_probs_decoder_ = Variable(log_probs_decoder.transpose(1, 2), requires_grad=True)
loss += F.cross_entropy(log_probs_decoder_.contiguous().
view((log_probs_decoder_.shape[1] * log_probs_decoder_.shape[0],
log_probs_decoder_.shape[2])),
targets.type(torch.long).view(-1),
ignore_index=model.num_classes-1)
```

When I comment out either of the criteria everything works. when I put it together It fails with the above comment. To simplify things I left only the ctc part of the network and replace the log_probs_decoder with torch.zeros_like(log_probs_ctc). still doesn’t work. I assume that the issue has to do with the two criterion competing somehow.

Thanks in advance for any help.