Extending AutoGrad from c++

When creating autograph extensions in C++ by subclassing the torch::autograd::Function

From the comments here, it seems that only direct Variable arguments are registered in the graph.

Is there any way I could pass a variable_list here? We are building a general interface for libtorch in R, and we only know the number of Variable argument at runtime. I see that python uses a different implementation here that uses a variable_list.

I’m afraid this is not possible with the API that we have for torch::autograd::Function.

There is always the solution of creating the Node by hand but this is going to be more complex (and outside of the official public API).

1 Like

Ok! Thanks!

Would you give us some pointer on how to implement the node by hand?
Would it just work If I implement a subclass of Node and implement it’s methods like:

and

I guess the best example would be gather/scatter in this file: https://github.com/pytorch/pytorch/blob/master/torch/csrc/autograd/functions/comm.cpp

The important bit is that Nodes are only applied. So you need one for the forward and one for the backward.
And you should use the wrap_outputs util from the functions/utils.cpp to properly set up the graph on your outputs before returning them.
Note that if you do inplace or views there is more stuff that you need to do (let me know if you need these).

1 Like

Thanks! Writing my own node worked. I mostly copied the code for CppNode and torch::Function removing the templates and accepting lambdas.

For other that may get here, here’s what we ended up writing.

See:

And here is an example call: