What is JIT "NNC" short for?

eval · December 14, 2020, 6:00am

Out of curiosity, anyone knows what the abbreviation “NNC” in JIT graph fuser means? Thanks.

tom · December 14, 2020, 11:34am

Neural Net Compiler:

pytorch/pytorch/blob/6e4de445010ef5e9e0438e4a4c16f6ef3129af14/test/cpp/tensorexpr/tutorial.cpp#L13


// This tutorial covers basics of NNC's tensor expressions, shows basic APIs to
// work with them, and outlines how they are used in the overall TorchScript
// compilation pipeline. This doc is permanently a "work in progress" since NNC
// is under active development and things change fast.
//
// This Tutorial's code is compiled in the standard pytorch build, and the
// executable can be found in `build/bin/tutorial_tensorexpr`.
//
// *** What is NNC ***
//
// NNC stands for Neural Net Compiler. It is a component of TorchScript JIT
// and it performs on-the-fly code generation for kernels, which are often a
// combination of multiple aten (torch) operators.
//
// When the JIT interpreter executes a torchscript model, it automatically
// extracts subgraphs from the torchscript IR graph for which specialized code
// can be JIT generated. This usually improves performance as the 'combined'
// kernel created from the subgraph could avoid unnecessary memory traffic that
// is unavoidable when the subgraph is interpreted as-is, operator by operator.
// This optimization is often referred to as 'fusion'. Relatedly, the process of
// finding and extracting subgraphs suitable for NNC code generation is done by

I’ve written a JIT tutorial on fusers (from a user perspective mostly looking into how things reach the fuser, hopefully online soon) and I standardized on calling them the TensorExpr fuser and the CUDA fuser.

Best regards

Thomas

eval · December 14, 2020, 8:20pm

This is very helpful. Thanks