Are all operations defined in torch.Tensor differentiable?

I’m relatively new to PyTorch, just wondering if I could apply any operation from torch.Tensor on a Variable and if it still would be differentiable. If not, is there a list of such functions that are not differentiable ?

all differentiable torch.* functions are enabled to be differentiable. Functions such as indexing (and a few more) cannot be differentiable wrt their indices and some inputs. When you try to differentiate such ones, an appropriate error is thrown.


Hello Soumith,

I have a related question regarding differentiability of some functions.

Is there any differentiable way to create a tensor of the size that is the value of some other tensor? In code:

size = torch.IntTensor([[2, 3]])
# One would expect that something like this:
#   tensor = torch.IntTensor(size)
# would work, but actually it is this:
width = size[0, 0]
height = size[0, 1]
tensor = torch.FloatTensor(height, width)

Is it even theoretically possible?

@mlajtos no because integer spaces are not differentiable (or continuous)

1 Like