This may be a fairly rare edge case, but wanted to confirm if this should be possible with the c++ api.
Suppose I have the following program calling backward of a different graph inside the backward hook of another one from a different thread.
I always need to call backward in a different thread because the main thread must be available to execute the hooks, that in my case go back to a an interface that only supports calls from the main thread.
#include <torch/torch.h>
#include <thread>
int main()
{
auto x = torch::randn(1, torch::requires_grad());
auto y = 2 * x;
std::function<void(torch::Tensor)> hook1 = [](torch::Tensor grad) {
std::cout << "hey" << std::endl;
};
x.register_hook(hook1);
auto a = torch::randn(1, torch::requires_grad());
auto b = 2 * a;
std::function<void(torch::Tensor)> hook2 = [&](torch::Tensor grad) {
std::cout << "hello" << std::endl;
std::thread t([&]() {
y.backward();
});
t.join();
};
a.register_hook(hook2);
std::thread t2([&]() {
b.backward();
});
// wait for hooks
t2.join();
return 0;
}