Why does faster-rcnn have a long delay(~10s) at the beginning?

I used pytorch to train a faster-rcnn model and then convert it into TorchScript model in order to deploy in c++ platform. After I deployed it in c++ platfrom, there is about 10s delay in predicting the second frame. so can anyone help me to solve it or give some valualbe comments? Thanks

This is when the JIT does its optimizations.
Typically, the first pass is an unoptimized profiling pass where the JIT collects tensor shape and layout information (and profiling means that, not something with timing). Then before the second pass, the JIT specializes the graph for the observed shapes and compiles fusion kernels etc. This is likely the delay you are seeing (though 10s seems a lot). After the second run (unless shapes change), you’ll be hitting the cached optimized path.

(That it is one profiling pass before the optimization kicks in is a configurable parameter.)

Some more details can be found e.g. on my blog posts on JIT optimization and the JIT runtime.

Best regards


Thanks your reply!
what you said is actually what I need.