Code Generation for PyTorch C++ Bare Metal (without OS)?

pkmital · October 21, 2018, 7:56pm

I’m interested in performing the code generation for running a pytorch model in C/C++. Specifically I would like this code to run on a bare metal device (no OS) and without any dependencies on things that an OS would typically provide such as threads or file IO. Further, it would be great to have this compile with as reduced a dynamic memory footprint as possible (as much static RAM usage as possible). Is this currently possible or a future goal of the Pytorch C++ frontend? Also are there any metrics on memory usage for loading pytorch header/libraries/dependencies in C++?

pkmital · October 21, 2018, 9:43pm

I just came across Glow which seems to do what I’m after?

github.com

pytorch/glow/blob/master/docs/AOT.md

# Creating standalone executable bundles

This document provides a short description about producing ahead-of-time
compiled executable bundles. The motivation for this work is to remove the cost
of compile time by allowing the users of Glow to compile the package ahead of
time.

## Overview

A bundle is a self-contained compiled network model that can be used to execute
the model in a standalone mode. After following the instructions in this
document and the [Makefile](../examples/bundles/resnet50) in the example
directory you will be able to compile convolutional neural networks into small
executables. Example:

```
  $make
  ...

  $cd build

This file has been truncated. show original

ptrblck · October 21, 2018, 9:44pm

I was about to post the same, but I wasn’t sure how “low level” you need to go.

Glow builds and runs on macOS and Linux. The software depends on a modern C++ compiler that supports C++11, on CMake, LLVM, protocol buffers, and libpng.