Serving pytorch trained model with Java server (Spring boot)

deployment
1

Is the pytorch provided Java library enough for serving (HTTP) a model in production?

Here are libraries used:

    implementation 'org.pytorch:pytorch_java_only:1.9.0'
    implementation group: 'com.facebook.fbjni', name: 'fbjni-java-only', version: '0.2.2'
    runtimeOnly group: 'com.facebook.soloader', name: 'nativeloader', version: '0.10.1'

    implementation group: 'org.springframework.boot', name: 'spring-boot', version: '2.5.6'

Can people share resources and suggestions for Java backed model serving in production?
I see there exists TorchServe written in Java but I don’t have control on Infrastructure to install torchserve.

2

Hi @ya_luyanta could you elaborate a bit more? You can pip install torchserve on any machine and serve models to it. I’d suspect you’ll need to install some library on a prod machine to serve models.