Does autograd differentiate with respect to the linspace arguments

autograd
1

The following program demonstrates that autograd does not differentiate with respect the argument to linspace:

import torch

start = torch.tensor( 0.0, requires_grad=True )
end   = torch.tensor( 1.0, requires_grad=True )
steps = torch.tensor( 2 )
#
x     = torch.linspace(start, end, steps, requires_grad=True)
y     = torch.stack( (start, end) )
#
print( "x = ", x )
print( "y = ", y )
#
x[0].backward()
print( "x[0].backward: start.grad = ", start.grad )
print( "x[0].backward: end.grad = ", end.grad )
#
start.grad = None
end.grad   = None
y[0].backward()
print( "y[0].backward: start.grad = ", start.grad )
print( "y[0].backward: end.grad = ", end.grad )
#
print( 'temp.py: OK' )

I get the following output for this program:

pytorch>python temp.py
x =  tensor([0., 1.], requires_grad=True)
y =  tensor([0., 1.], grad_fn=<StackBackward0>)
x[0].backward: start.grad =  None
x[0].backward: end.grad =  None
y[0].backward: start.grad =  tensor(1.)
y[0].backward: end.grad =  tensor(0.)
temp.py: OK
pytorch>vi temp.py
pytorch>python temp.py
x =  tensor([0., 1.], requires_grad=True)
y =  tensor([0., 1.], grad_fn=<StackBackward0>)
x[0].backward: start.grad =  None
x[0].backward: end.grad =  None
y[0].backward: start.grad =  tensor(1.)
y[0].backward: end.grad =  tensor(0.)
temp.py: OK
pytorch>
2

Hi Brad!

Yes, this is correct.

Yes, there is indeed no autograd connection between your x and start (nor end).

(Note that steps is naturally an integer, so you would not expect linspace() to be
differentiable with respect to steps.)

If you need linspace() to be differentiable with respect to start and end, a sensible
approach would be to use linspace() to create a (non-differentiable) tensor that runs
from 0.0 to 1.0 and then linearly scale that result to run from start to end (where
start and end carry requires_grad = True).

Best.

K. Frank

3

@KFrank Thanks for the reply. I didn’t notice any mention of differentiation in the linspace documentation for start, end, or steps; see

https://docs.pytorch.org/docs/2.13/generated/torch.linspace.html#torch-linspace

I know that exp supports differentiation with respect to its argument but it’s documentation does not mention differentiation; see

https://docs.pytorch.org/docs/2.13/generated/torch.exp.html#torch.exp

So in general, the documentation does not cover which arguments support differentiation ?