Exhaustive list of environment variables for PT2.0 Compiler Debug Traces

Abhishek_Ghosh · July 8, 2024, 5:41am

I often struggle to find out which environment variable would enable certain debug traces. Often, I find myself looking into the source code to figure out the environment variable check hidden behind a conditional to enable the debug tracing.

I have seen certain useful environment variables mentioned in PyTorch blogs/docs (the troubleshooting guide, for example) or tutorials/talks. But they are not exhaustive.

For instance, I want to print the “cache hit” message from _custom_eval_frame:

github.com

pytorch/pytorch/blob/c5c9dbece10e92e325e4efd0aea91d681737142d/torch/csrc/dynamo/eval_frame.c#L620


      
            if (maybe_cached_code == NULL) {
              // guard eval failed, keep propagating
              *should_clear_frame = 1;
              return NULL;
            } else if (maybe_cached_code == Py_None) {
              DEBUG_TRACE("cache miss %s", get_frame_name(frame));
              return eval_frame_default(tstate, frame, throw_flag);
            }
            PyCodeObject* cached_code = (PyCodeObject*)maybe_cached_code;
            // used cached version
            DEBUG_TRACE("cache hit %s", get_frame_name(frame));
            *should_clear_frame = 1;
            return eval_custom_code(tstate, frame, cached_code, throw_flag, free_vars_copied);
          }
          DEBUG_CHECK(PyDict_CheckExact(frame->f_locals));
          DEBUG_CHECK(PyDict_CheckExact(frame->f_globals));
          DEBUG_CHECK(PyDict_CheckExact(frame->f_builtins));
          
          // We don't run the current custom_eval_frame behavior for guards.
          // So we temporarily set the callback to Py_None to drive the correct behavior
          // in the shim.

Do you have any suggestions for this?

The above specific case is a compilation flag, which is enabled by uncommenting #define TORCHDYNAMO_DEBUG 1 in torch/csrc/dynamo/debug_macros.h