Pytorch not compatible with React-native library

Arun_Kumar1 · March 20, 2020, 5:27am

I’ve been using react native for one of my module.Recently we started using Pytorch . Pytorch along with react native is crashing.
Below is the error

java.lang.UnsatisfiedLinkError: couldn’t find DSO to load: libpytorch_jni.so caused by: dlopen failed: cannot locate symbol “_ZN8facebook3jni10JByteOrder11nativeOrderEv” referenced by “/data/app/*****-4ZGDZJcLItKdhIsz3hj2rQ==/lib/arm64/libpytorch_jni.so”.

Please help me.I am in great urgency.

IvanKobzarev · March 22, 2020, 3:26pm

Hello, @Arun_Kumar1

Could you please share your build setup: gradle files, CMakeLists.txt ( if you have it)?

Both react native and pytorch android use fbjni library, while “_ZN8facebook3jni10JByteOrder11nativeOrderEv” was added for pytorch android, it looks like system loads first react native libfbjni.so that does not have JByteOrder::nativeOrder method which is needed for libpytorch_jni.so

Arun_Kumar1 · March 23, 2020, 5:08am

Hi @IvanKobzarev,

Please check the below build.gradle file content.I don’t have CMakeLists.txt.

apply plugin: “com.android.application”

import com.android.build.OutputFile

/**

The react.gradle file registers a task for each build variant (e.g. bundleDebugJsAndAssets
and bundleReleaseJsAndAssets).
These basically call react-native bundle with the correct arguments during the Android build
cycle. By default, bundleDebugJsAndAssets is skipped, as in debug/dev mode we prefer to load the
bundle directly from the development server. Below you can see all the possible configurations
and their defaults. If you decide to add a configuration block, make sure to add it before the
apply from: "../../node_modules/react-native/react.gradle" line.
project.ext.react = [
// the name of the generated asset file containing your JS bundle
bundleAssetName: “index.android.bundle”,
// the entry file for bundle generation
entryFile: “index.android.js”,
// whether to bundle JS and assets in debug mode
bundleInDebug: false,
// whether to bundle JS and assets in release mode
bundleInRelease: true,
// whether to bundle JS and assets in another build variant (if configured).
// See http://tools.android.com/tech-docs/new-build-system/user-guide#TOC-Build-Variants
// The configuration property can be in the following formats
// ‘bundleIn${productFlavor}${buildType}’
// ‘bundleIn${buildType}’
// bundleInFreeDebug: true,
// bundleInPaidRelease: true,
// bundleInBeta: true,
// whether to disable dev mode in custom build variants (by default only disabled in release)
// for example: to disable dev mode in the staging build type (if configured)
devDisabledInStaging: true,
// The configuration property can be in the following formats
// ‘devDisabledIn${productFlavor}${buildType}’
// ‘devDisabledIn${buildType}’
// the root of your project, i.e. where “package.json” lives
root: “…/…/”,
// where to put the JS bundle asset in debug mode
jsBundleDirDebug: “$buildDir/intermediates/assets/debug”,
// where to put the JS bundle asset in release mode
jsBundleDirRelease: “$buildDir/intermediates/assets/release”,
// where to put drawable resources / React Native assets, e.g. the ones you use via
// require(’./image.png’)), in debug mode
resourcesDirDebug: “$buildDir/intermediates/res/merged/debug”,
// where to put drawable resources / React Native assets, e.g. the ones you use via
// require(’./image.png’)), in release mode
resourcesDirRelease: “$buildDir/intermediates/res/merged/release”,
// by default the gradle tasks are skipped if none of the JS files or assets change; this means
// that we don’t look at files in android/ or ios/ to determine whether the tasks are up to
// date; if you have any other folders that you want to ignore for performance reasons (gradle
// indexes the entire tree), add them here. Alternatively, if you have JS files in android/
// for example, you might want to remove it from here.
inputExcludes: [“android/", "ios/”],
// override which node gets called and with what additional arguments
nodeExecutableAndArgs: [“node”],
// supply additional arguments to the packager
extraPackagerArgs: []
]
*/

project.ext.react = [
entryFile: “index.js”,
enableHermes: true, // HERE!
]

apply from: “…/…/node_modules/react-native/react.gradle”

/**

Set this to true to create two separate APKs instead of one:
- An APK that only works on ARM devices
- An APK that only works on x86 devices
The advantage is the size of the APK is reduced by about 4MB.
Upload all the APKs to the Play Store and people will download
the correct one based on the CPU architecture of their device.
*/
def enableSeparateBuildPerCPUArchitecture = false

/**

Run Proguard to shrink the Java bytecode in release builds.
*/
def enableProguardInReleaseBuilds = false

android {
compileSdkVersion rootProject.ext.compileSdkVersion
buildToolsVersion ‘28.0.3’

defaultConfig {
    applicationId "com.progressreport"
    minSdkVersion rootProject.ext.minSdkVersion
    targetSdkVersion rootProject.ext.targetSdkVersion
    versionCode 1
    versionName "1.0"
    ndk {
        abiFilters 'x86','armeabi-v7a','arm64-v8a','x86_64'
    }
}

packagingOptions {
    pickFirst "**/libfbjni.so"

}
splits {

    // Configures multiple APKs based on screen density.
    splits {
        abi {
            enable true
            reset()
            include 'armeabi-v7a'
            universalApk false
        }
    }
}
buildTypes {
    release {
        minifyEnabled enableProguardInReleaseBuilds
        proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
    }
}
compileOptions {
    sourceCompatibility 1.8
    targetCompatibility 1.8
}
packagingOptions {
    pickFirst '/armeabi-v7a/libc++_shared.so'
    pickFirst '/x86/libc++_shared.so'
    pickFirst '/arm64-v8a/libc++_shared.so'
    pickFirst '/x86_64/libc++_shared.so'
    pickFirst '/x86/libjsc.so'
    pickFirst '/armeabi-v7a/libjsc.so'
    pickFirst '/arm64-v8a/libpytorch_jni.so'
}
// applicationVariants are e.g. debug, release
applicationVariants.all { variant ->
    variant.outputs.each { output ->
        // For each separate APK per architecture, set a unique version code as described here:
        // http://tools.android.com/tech-docs/new-build-system/user-guide/apk-splits
        def versionCodes = [ "x86":1,"x86_64":2,"armeabi-v7a":3,"arm64-v8a":2]
        def abi = output.getFilter(OutputFile.ABI)
        if (abi != null) {  // null for the universal-debug, universal-release variants
            output.versionCodeOverride =
                    versionCodes.get(abi) * 1048576 + defaultConfig.versionCode
        }
    }
}

}

dependencies {
implementation project(’:react-native-svg’)
implementation project(’:lottie-react-native’)
implementation project(’:react-native-localization’)
implementation project(’:react-native-linear-gradient’)
implementation project(’:react-native-vector-icons’)
implementation project(’:react-native-gesture-handler’){}
implementation fileTree(dir: “libs”, include: ["*.jar"])

implementation "com.android.support:appcompat-v7:${rootProject.ext.supportLibVersion}"
implementation ("com.facebook.react:react-native:+" ){
    exclude  module: 'fbjni-java-only'
} // From node_modules
def jscFlavor = 'org.webkit:android-jsc:+'
def enableHermes = project.ext.react.get("enableHermes", false);
if (enableHermes) {
    // for RN 0.61+
    def hermesPath = "$rootDir/../node_modules/hermes-engine/android/";
    debugImplementation files(hermesPath + "hermes-debug.aar")
    releaseImplementation files(hermesPath + "hermes-release.aar")
} else {
    implementation jscFlavor
}
def useIntlJsc = false
if (useIntlJsc) {
    implementation ('org.webkit:android-jsc-intl:+')
} else {
    implementation ('org.webkit:android-jsc:+')
            {
                exclude group:'com.facebook.fbjni',module:  "fbjni-java-only-0.0.3"
            } // From node_modules/ From node_modules

}
implementation(name:'pytorch_android-1.5.0-SNAPSHOT', ext:'aar')
implementation(name:'pytorch_android_fbjni-1.5.0-SNAPSHOT', ext:'aar')
implementation 'com.facebook.soloader:nativeloader:0.8.0'


implementation "androidx.multidex:multidex:2.0.1"

}

// Run this once to be able to run the application with BUCK
// puts all compile dependencies into folder libs for BUCK to use
task copyDownloadableDepsToLibs(type: Copy) {
from configurations.compile
into ‘libs’
}

IvanKobzarev · March 24, 2020, 10:33pm

Hi @Arun_Kumar1,
I have reproduced your problem. We have contacted react-native team to align our libfbjni.so’s. It might take some time and updating to react-native master.

If you desperately need to unblock yourself you could try hacky way to use libfbjni.so from pytorch-android in react-native.

Downloading nightly build of pytorch android that contains libfbjni.so inside,

wget -O pa.aar https://oss.sonatype.org/service/local/repositories/snapshots/content/org/pytorch/pytorch_android/1.5.0-SNAPSHOT/pytorch_android-1.5.0-20200324.090131-79.aar

# We will need absolute path to downloaded file
PA_AAR_ABSOLUTE_PATH=$(pwd)/pa.aar

Edit node_modules/react-native/ReactAndroid/build.gradle (https://github.com/facebook/react-native/blob/master/ReactAndroid/build.gradle#L368) changing task extractJNIFiles which is at the moment is used only for fbjni

copy {
    from zipTree(file)
    into "$projectDir/src/main/jni/first-party/$packageName/"
    include "jni/**/*"
}

to

copy {
    from zipTree(new File("${PA_AAR_ABSOLUTE_PATH}"))
    into "$projectDir/src/main/jni/first-party/$packageName/"
    include "jni/**/libfbjni.so"
}

where new File(…) argument is absolute path to pa.aar that was downloaded on step 1.
It worked for me.
(The main idea of this hack - to use libfbjni.so from pytorch-android)

I will reply here when react-native and pytorch-android libfbjni will be aligned.
Sorry for inconvenience.

Arun_Kumar1 · March 25, 2020, 5:55am

Thanks @IvanKobzarev for quick followup.
Will follow the same steps as mentioned to unblock ourselves for now.

HugoGresse · August 10, 2020, 8:55am

Hi, is there an update on the .so alignment between RN & PyTorch? Or an issue to follow?
I wanted to use PyTorch from RN but I’m blocked and don’t really want to force a fbjni that may cause issue in production for RN.

HugoGresse · August 18, 2020, 2:10pm

After some more investigation, using the release 1.5.0 android, I was able to get the app to compile but not run the PyTorch. So I got almost the same issue but with a slightly different error message:

E/SoLoader: couldn't find DSO to load: libpytorch_jni.so caused by: dlopen failed: 
cannot locate symbol "_ZN8facebook3jni11JByteBuffer5orderENS0_9alias_refINS0_10JByteOrderEEE" 
referenced by "/data/app/com.offline-EWIa3xVhdQYckXATRsmgrw==/lib/arm64/libpytorch_jni.so"... result: 0

The symbol of _ZN8facebook3jni11JByteBuffer5orderENS0_9alias_refINS0_10JByteOrderEEE differ from _ZN8facebook3jni10JByteOrder11nativeOrderEv of the original question.

When trying the proposed solution (which is to get the fbjni from PyTorch rather than RN one, it produced the same error

Arun_Kumar1 · August 18, 2020, 2:32pm

@HugoGresse Please upgrade RN to latest version.If you are using custom build Pytorch library.
Please do the following.

implementation(“com.facebook.react:react-native:0.62.2”){
exclude module: ‘fbjni-java-only’
}

duclh3 · August 23, 2020, 9:08am

@Arun_Kumar1
I make an upgrading to RN 0.62.2, but I still have the same error as @HugoGresse mentioned.
Do you have another way to fix this bug ?
Sorry for my bad English!

Sxela · August 28, 2020, 11:57am

Guys, importing pytorch like that seems to work for me:

    implementation ('org.pytorch:pytorch_android:1.5.0'){
     exclude module: 'fbjni-java-only'
}
    implementation ('org.pytorch:pytorch_android_torchvision:1.5.0'){
     exclude module: 'fbjni-java-only'
}
    implementation ('org.pytorch:pytorch_android_fbjni:1.5.0'){
     exclude module: 'fbjni-java-only'
}
    implementation('com.facebook.react:react-native:+'){
     exclude module: 'fbjni-java-only'
}

I also have

android {
        packagingOptions {
	pickFirst '**/*.so'
}
}

so that it won’t go crazy over those multiple libfbjni.so I’m getting for some reason. Hope that helps!

CodeSammich · December 16, 2020, 4:33am

This worked for me! Thank you!!!