CPP Simple TorchVision example

C++
1

Hello,

I am trying to compile something that simply initializes a random torchvision.models model, like ResNet50, and then just passes a simple random tensor through. All the examples that I have seen so far involve loading the model from the disk rather than being able to just initialize it out of the box like in Python. Specifically, in C++ the examples so far instruct the following:

#include <torch/script.h> // One-stop header.

#include <iostream>
#include <memory>

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr << "usage: example-app <path-to-exported-script-module>\n";
    return -1;
  }

  // Deserialize the ScriptModule from a file using torch::jit::load().
  std::shared_ptr<torch::jit::script::Module> module = torch::jit::load(argv[1]);

  assert(module != nullptr);
  std::cout << "ok\n";
}

However, in Python I can just do this:

import torch
from torchvision import models

resnet50 = models.resnet50(pretrained=False)

Hence, in Python, there was no need to load any file from the disk to initialize a simple ResNet model. I am also looking for a way to initialize a model in C++ similarly. Does anyone know anyway (even with third-party tools that are compatible with PyTorch C++ API)?

2

@krshrimali published a few blog posts here showing how the C++ API can be used. I don’t know if these tutorials are still up to date but they could be a good starter.

3

Thank you, Patric, for your prompt response. I visited the links. Even though at first they seem to be down, one can quickly search through the older post of the creator and find the tutorials. In the creator’s tutorials, the model classes (e.g., VGG-16) are created from scratch in CPP. In a different tutorial, the user also loads the model from a pre-built torch.jit checkpoint. Furthermore, I have experienced cases where torch.jit does not compile the model properly. For example, in the case of AlexNet, if one builds the model using torch.jit and then attempts to perform inference, they will observe that when the parser reaches the linear layers, forward passing will fail because the flatten() operation is not embedded in the jit checkpoint. Another interesting addition to this topic is that this solution is not scalable if one targets workloads of multiple DNNs running concurrently. I also posted a question on GitHub, and it seems that, for some reason, the native PyTorch CPP implementations were deprecated. I also expressed my opinion on why it is useful for many use cases to reinstate the native CPP model implementations (at list for torchvision.models).

1 Like
4

Making minor corrections to my response. Jit does successfully parse the hidden tensor operations (operations that are not nn.Module() classes like a.flatten()). I think my intel was outdated, and now that I have run some tests again, it seems to work alright. Still, I think the original issue is unresolved.