Dense stn (Spatial Transformer Networks)

Hi, I am trying to implement stn. The affine_grid in torch.nn supports global transformation (for the whole image), i.e theta is Nx2x3 but I am looking for an implementation where each pixel has a 2x3 affine transformation. So theta in my case would be NxHxWx2x3.

Any help would be appreciated!

Have you seen this tutorial on PyTorch website for STN: https://pytorch.org/tutorials/intermediate/spatial_transformer_tutorial.html? There is an example showing how to implement STN for MNIST images.

Yes. I have seen it but it uses “affine_grid” which supports global transformation only (Nx2x3)

Oh, so you want to transform each pixel independently! Sorry, I do not have more experience beyond this, but also I cannot imagine what would happen to an image if each pixel is transformed separately!

That’s right. I want to transform each pixel independently.

Update: I was looking for this. Thanks!