Filling a 2D tensor at specific indices without detaching

Here’s my problem: I’m trying to fit a series of splines to a pixel image of wavy lines that go from left to right. I’ve found an autograd-friendly implementation of splines, then my plan was to reconstruct an image from the splines and pass the generated output and a target to the loss function.

For the optimization to work, the whole chain needs to be differentiable. However I can’t find a way of reconstructing the images from the splines in a way that doesn’t detach the chain.

I could take the spline and turn the values into x,y coords to set certain elements of a tensor, but when I int() them it detaches of course.

I found a grid_sample, which is a differentiable version of the exact opposite of what I need to do. I need to fill certain points in a tensor using float indices and interpolation.

Basically I need to “paint” into a tensor in an autograd-friendly way.

I’m a bit lost, any hints would be super useful.

Ah, I figured it out.
Basically you can floor the float index, then interpolate the int indices using the float one. That’s autograd friendly.

Hello @robclouth,

I want to do similar work as you mentioned. I want to learn some parameters to paint an image in some positions (using some indexes) to maximize the classification performance of the images. I tried to understand the floor(idx) and interpolation. However, I could not figure out how I to access the positions in the image for changing the RGB channel of the image.
In the mentioned code. I could not relate the output tensor to the image I want to change the color values.
I appreciate it if you help me!