catphive
(Brendan Miller)
December 27, 2018, 7:14am
1
I’m trying to get a handle of the organization the pytorch code base.
I see in an earlier version where matrix inverse and the backward operation for matrix inverse are implemented:
import torch
from ..function import Function
class Diag(Function):
def __init__(self, diagonal_idx=0):
super(Diag, self).__init__()
self.diagonal_idx = diagonal_idx
def forward(self, input):
return input.diag(self.diagonal_idx)
def backward(self, grad_output):
return grad_output.diag(self.diagonal_idx)
class Tril(Function):
This file has been truncated. show original
However, in the new 1.0 version, I can find the implementation for inverse in BatchLinearAlgebra.cpp (https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/BatchLinearAlgebra.cpp ), but I can’t figure out where the implementation of backwards for matrix inverse is.
Can someone give me a hint where this and other other implementations of backwards are located? The organization of the C++ part of the code base is a bit obscure.
smth
December 28, 2018, 11:37pm
2
The backward of inverse is not implemented, because all the functions it calls have backward implemented on themselves.
So effectively, it’s similar to how lambda x: torch.add(x, 1).mul(2)
does not need backward implemented on it, because .add
and .mul
have backward defined on them.
catphive
(Brendan Miller)
December 30, 2018, 4:06am
3
It looks like inverse just calls lapack routines. I don’t think it’s calling other pytorch function to implement matrix inverse.
I’m looking at apply_inverse in BatchLinearAlgebra.cpp:185.
smth
December 30, 2018, 4:29am
4
I took you down the wrong rabbit-hole and its ENTIRELY my fault, sorry about that.
We do have functions in C+±land whose derivatives are entirely defined by tracing, but this is not such a function.
inverse
's derivate is speficied in derivatives.yaml here: https://github.com/pytorch/pytorch/blob/ef487d4f1d7b167dc28976b90e3a728006f37a55/tools/autograd/derivatives.yaml#L396-L397
catphive
(Brendan Miller)
December 30, 2018, 5:04pm
5
Thanks! Great. I see. So a bunch of code is being generated from YAML files.