I have a collection of feature maps, e.g., 4D tensor, and a list of 2D coordinates. I want to extract feature vectors at these coordinates, and use them as input to the next layer. In order not to break autograd, how should I implement it?
My naive idea is to write a for loop and use indexing to extract 2D tensors at each location, but I don’t know whether it will break autograd. Is there any existing function that can performs the similar task?