Custom cpp/cuda extensions - debugging

HjalmarLucius · April 9, 2018, 12:02am

Hi, I’m trying to learn making my own 0.4a pytorch extensions by following the excellent tutorial of Goldsborough (link below). I’ve got things working but I can’t find a way to debug things when doing work myself (current workflow is simply compile with jit, run some commands and see if it works).

Is it possible to compile and debug the ATen .cu / .cpp files? I tried using nvcc but was unable to work out how to run things.

It would probably be of great help to have a brief section in the tutorial on this as well.

Disclaimer: First time I’m coding either cpp or CUDA

Link to tutorial:

github.com

goldsborough/tutorials/blob/cpp-extension/advanced_source/cpp_extension.rst

Custom C++ and CUDA Extensions
==============================
**Author**: `Peter Goldsborough <http://goldsborough.me>`_


PyTorch provides a plethora of operations related to neural networks, arbitrary
tensor algebra, data wrangling and other purposes. However, you may still find
yourself in need of a more customized operation. For example, you might want to
use a novel activation function you found in a paper, or implement an operation
you developed as part of your research.

The easiest way of integrating such a custom operation in PyTorch is to write it
in Python by extending :class:`Function` and :class:`Module` as outlined `here
<http://pytorch.org/docs/master/notes/extending.html>`_. This gives you the full
power of automatic differentiation (spares you from writing derivative
functions) as well as the usual expressiveness of Python. However, there may be
times when your operation is better implemented in C++. For example, your code
may need to be *really* fast because it is called very frequently in your model
or is very expensive even for few calls. Another plausible reason is that it
depends on or interacts with other C or C++ libraries. To address such cases,

This file has been truncated. show original