Android pytorch forward() method running in a separate thread slow down UI thread

Is that possible to find out the reason which caused lagging of UI like specific thread or method?

Cause I kind of put all the heavy work into another thread but I’ve still getting a lagged UI.

I’m using CameraX where ImageAnalysis works in separate thread in which I do emotion detection via neural network. So when a neural network process an image in this not UI thread my UI thread also lags.

Is that possible at all? Can the separate thread slow down the UI thread by executing heavy task?

Maybe there’s some plugin for Android Studio or something like that to solve the problem.

I will be glad to any advice…

Hello @Chame_call.
At the moment pytorch android thread count is fixed by device and equal “number of big cores of cpu”(N) on device.
As you also have bg thread for image decoding + UI thread - at the moment of inference you have at least (N + 2) competing threads for N big cores (+other applications threads). I think that is the reason why you see UI thread slow downs.

We are thinking to expose control of number of threads to java api, in that case you can set for example singleThread mode, that should not affect too much UI thread responsiveness.

For more details I would recommend to use android systrace.
If you are building pytorch android from the source you can do deeper investigation with systrace if you build it with environment variable TRACE_ENABLED=1
(https://github.com/pytorch/pytorch/blob/master/android/pytorch_android/CMakeLists.txt#L7)
It controls logging of additional sections for systrace, in that case you will see operators sections.

If you need all debug symbols for tracing - you may check example of test_app in our repo, that has a script how to build it with all c++ debug symbols:

TRACE_ENABLED=1 sh android/build_test_app.sh
1 Like

Hello @Chame_call

We just exposed conrtol on global number of threads used by pytorch android, it was landed in master

method org.pytorch.Module#setNumThreads(int numThreads)

(https://github.com/pytorch/pytorch/blob/master/android/pytorch_android/src/main/java/org/pytorch/Module.java#L57)
The latest android nightlies already include them: https://github.com/pytorch/pytorch/tree/master/android#nightly (you might need gradle argument --refresh-dependencies if you already using them)

For your case when inference is slowing down UI thread - you can experiment with setting number of threads to 1 or 2, maybe depending on device num cores, smth like:

Module module = Module.load(moduleFileAbsoluteFilePath);
module.setNumThreads(1);

I think that should help with UI thread responsiveness, but check if inference time with this setting is still acceptable for your solution.

This is new functionality, please report if you find any issues with it.

1 Like

Update:

We moved setNumThreads method to separate class org.pytorch.PyTorchAndroid as a static method.

1 Like