Conv2d/conv3d at only selected pixel locations

kBiren · March 6, 2019, 10:46pm

Is it possible to perform convolution operation in pytorch at specific locations (list of pixel location probably fed by the user or collected from other algorithm) rather than at locations predetermined by the stride?

InnovArul · March 6, 2019, 11:00pm

I am not sure if it is possible yet. As a naive solution (assuming that computation is not a bottleneck), why can’t you multiply the conv output with a mask that specifies 1 in the locations selected by user or other algorithms?

kBiren · March 6, 2019, 11:55pm

The main goal is to avoid unnecessary computations, make algorithm memory friendly and speed up the process. Assume you have a bunch of points in 3D space and you want to operate convolution with nxnxn filter at those points but not in grid locations where there are no points.

Thanks for your reply.

kBiren · March 11, 2019, 8:47am

I found the solution.

gist.github.com

https://gist.github.com/lantiga/a78581e6c6c0ad1534065950e204ce9d

README.md


# Indexed convolutions

A convolution operator over a 1D tensor (BxCxL), where a list of neighbors for each element is provided through a indices tensor (LxK), where K is the size of the convolution kernel. Each row of indices specifies the indices of the K neighbors of the corresponding element in the input. A -1 is handled like for zero padding.

Note that the neighbors specified in indices are not relative, but rather absolute. They have to be specified for each of the elements of the output.

A use case is for convolutions over non-square lattices, such as images on hexagonal lattices coming from Cherenkov telescopes (http://www.isdc.unige.ch/%7Elyard/FirstLight/FirstLight_slowHD.mov).

Example:

This file has been truncated. show original

indexed_conv.py

import torch
from torch.autograd import Variable

def prepare_mask(indices):

    padded = indices == -1
    indices[padded] = 0

    mask = torch.FloatTensor([1,0])
    mask = mask[..., padded.t().long()]

This file has been truncated. show original