Are all operations defined in torch.Tensor differentiable?

1

I’m relatively new to PyTorch, just wondering if I could apply any operation from torch.Tensor on a Variable and if it still would be differentiable. If not, is there a list of such functions that are not differentiable ?

2

all differentiable torch.* functions are enabled to be differentiable. Functions such as indexing (and a few more) cannot be differentiable wrt their indices and some inputs. When you try to differentiate such ones, an appropriate error is thrown.

3 Likes
3

Hello Soumith,

I have a related question regarding differentiability of some functions.

Is there any differentiable way to create a tensor of the size that is the value of some other tensor? In code:

size = torch.IntTensor([[2, 3]])
print(size)
# One would expect that something like this:
#   tensor = torch.IntTensor(size)
# would work, but actually it is this:
width = size[0, 0]
height = size[0, 1]
tensor = torch.FloatTensor(height, width)
print(tensor)

Is it even theoretically possible?

4

@mlajtos no because integer spaces are not differentiable (or continuous)

1 Like