Here’s my problem: I’m trying to fit a series of splines to a pixel image of wavy lines that go from left to right. I’ve found an autograd-friendly implementation of splines, then my plan was to reconstruct an image from the splines and pass the generated output and a target to the loss function.
For the optimization to work, the whole chain needs to be differentiable. However I can’t find a way of reconstructing the images from the splines in a way that doesn’t detach the chain.
I could take the spline and turn the values into x,y coords to set certain elements of a tensor, but when I int() them it detaches of course.
I found a grid_sample, which is a differentiable version of the exact opposite of what I need to do. I need to fill certain points in a tensor using float indices and interpolation.
Basically I need to “paint” into a tensor in an autograd-friendly way.
I want to do similar work as you mentioned. I want to learn some parameters to paint an image in some positions (using some indexes) to maximize the classification performance of the images. I tried to understand the floor(idx) and interpolation. However, I could not figure out how I to access the positions in the image for changing the RGB channel of the image.
In the mentioned code. I could not relate the output tensor to the image I want to change the color values.
I appreciate it if you help me!