Grid_Sample loop instead of full blocks

Hey all,

Previously, I was using F.grid_sample to process a block of images simultaneously. However, due to the data sizes, I’m moving towards wanting to process each of the images separately so that I don’t have memory size problems on the GPU.

Previously, I was doing something like

    frame_data = grid_sample(frame_data, row_col_grid, mode='bilinear', padding_mode='zeros')

frame_data is a <1, 50, 1024, 1024> Tensor and row_col_grid is a <1, 1024, 1024, 2> Tensor.

That was working, other than potential gpu memory issues in the future with larger images.

Now, I’m trying to move to something like the following:

    for frame in frame_data:
        frame = frame.unsqueeze(0).unsqueeze(0).to('cuda:0')
        frame = grid_sample(frame, row_col_grid, mode='bilinear', padding_mode='zeros')

When frame hits grid_sample, it is size <1, 1, 1024, 1024> with row_col_grid still <1, 1024, 1024, 2>.

One would think that I’d get the same list of frames at the end, but for some reason, I’m getting frames with only zeros (which I understand to be the padding, I believe).

Any thoughts on how to loop through these images instead of processing them in a block?

(Also, if it wasn’t obvious, I’m using the PyTorch 1.3 with Python 3.7.)