How to Export `torchaudio.models.decoder.ctc_decoder` to TorchScript for C++ Deployment?

audio
1

Hello everyone,

I’m trying to export torchaudio.models.decoder.ctc_decoder to TorchScript so that I can use it in a C++ application. However, I have encountered several issues along the way.

What I Tried

  1. Using torch.jit.script()

    • Running torch.jit.script(decoder) gives the following error:
      torch.jit.frontend.UnsupportedNodeError: Lambda aren't supported
    • The issue seems to be related to a lambda function inside the _get_tokens method of CTCDecoder, which TorchScript does not support.

  2. Using torch.jit.trace()

    • I tried tracing the decoder using:
      scripted_decoder = torch.jit.trace(decoder, dummy_input)
    • This resulted in the following error:
      RuntimeError: Tracer cannot infer type of [[CTCHypothesis(tokens=tensor([...]))]]
    • The problem here is that the decoder returns a list of CTCHypothesis objects, which TorchScript cannot process.

  3. Wrapping the Decoder in nn.Module

    • I attempted to wrap the decoder inside an nn.Module subclass to make it compatible with scripting, but the same CTCHypothesis type inference error occurred.

Question

  • Is there any recommended way to export torchaudio.models.decoder.ctc_decoder to TorchScript?
  • If not, what would be the best approach to deploy it in a C++ application?
  • Would modifying the source code of CTCDecoder (e.g., replacing lambda functions and using standard PyTorch types) be a feasible solution?

Any guidance or workarounds would be greatly appreciated!

Thanks in advance!

@ptrblck @smth @albanD @carolineechen

2

Hi @harald_scheidl !

Do you have any idea?