Here’s my problem: I’m trying to fit a series of splines to a pixel image of wavy lines that go from left to right. I’ve found an autograd-friendly implementation of splines, then my plan was to reconstruct an image from the splines and pass the generated output and a target to the loss function.

For the optimization to work, the whole chain needs to be differentiable. However I can’t find a way of reconstructing the images from the splines in a way that doesn’t detach the chain.

I could take the spline and turn the values into x,y coords to set certain elements of a tensor, but when I int() them it detaches of course.

I found a grid_sample, which is a differentiable version of the exact opposite of what I need to do. I need to fill certain points in a tensor using float indices and interpolation.

Basically I need to “paint” into a tensor in an autograd-friendly way.

I’m a bit lost, any hints would be super useful.