Custom node parsing

Hi,
We currently develop a Glow backend for CEVA target DSP that supports AOT (offline) compilation mode.
We would like to be able to compile a model containing layers which glow does not currently support, for example, the Loop layer, so that we could offload them to the DSP.
As far as I understand, the model parsing is done in the frontend.
Is there a way to add custom model parsing for a specific layer by the backend, such that it’d be automatically called during the parsing stage, without modifying the frontend code?
Thanks.

Hi @Arkady – we have currently have an ExternalFunctionCall node which is capable of representing a node that only a specific backend understands how to handle/execute. There is also a more extensive/feature rich version of this same idea in this PR but not sure when it will land.

Thanks for the reply.
However, it looks like both the ExternalFunctionCall node and the enhancement of this feature, as described in the link you mentioned, add the possibility to use custom code during execution.
But how can I trigger custom layer parsing during the parsing of the model?

For example:
I’m trying to compile a bundle from a tensorflow-lite model which contains the “SpaceToBatch” layer, which isn’t supported by glow, and as a result I receive the following error:
“Error message: TensorFlowLite: Operator type ‘SPACE_TO_BATCH_ND’ is not supported!”

But the backend knows how to parse this layer - When this layer is encountered, I’d like to create a new MySpace2BatchNode (which is defined in the backend), and add it to the graph.
Is there a way to instruct glow (without frontend code changes), that if the model parser encountered a layer it cannot parse, ask the backend if it can parse it, instead of failing immediately?
In other words, can the control pass to the backend before the parsing of the model is complete, or is the only way to change the behavior of the parser is to modify the parser code itself?
Thanks.

Got it.

Is there a way to instruct glow (without frontend code changes), that if the model parser encountered a layer it cannot parse, ask the backend if it can parse it, instead of failing immediately?

Depends by what you mean without frontend code changes. The current master cannot do this. We would need to make changes to do something like, if the node isn’t supported in Glow IR, then check the backend if it supports it as an ExternalFunctionCall or something similar, and if so load it as such. I.e. this would require a single code change to the frontend so that such layers could be loaded as something like an ExternalFunctionCallNode. It would then be possible to load them and then the backend would be responsible for its execution.