How to covert optical flow to tensor?

vision
1

Sorry to bother you , I read the document and

torchvision.transforms.ToTensor : Converts a PIL Image or numpy.ndarray (H x W x C) in the range [0, 255] to a torch.FloatTensor of shape (C x H x W) in the range [0.0, 1.0].

well, my flow matrix various from negative number to positive number, I use cv2.calcOpticalFlowFarneback fuction to calculate it, how can I fix this?
Thanks for your reply!

2

you can use torch.from_numpy() stament.
https://pytorch.org/docs/stable/torch.html#torch.from_numpy

This allows you to convert numpy arrays into tensors.

1 Like
3

Many thanks! What’ more, the flow matrix varied a lot ,so the mean value of channel doesn’t convergence to a certain value, so how can I normalize this matrix?
Thanks!

4

it’s a difficult question in case of optical flow.
First of all, depending on which net are you using normalization you need vary a lot. If you are using both displacement components you have to apply the same mapping for both, not to affect the vector. You could use just the magnitude.

You should check some paper about optical flow normalization. Btw you can try mapping the whole OF range into [0,1] or [-1,1]

def rescale(x,max_range,min_range):
    max_val = np.max(x)
    min_val = np.min(x)
    return (max_range-min_range)/(max_val-min_val)*(x-max_val)+max_range

Consider you have to do this operation to the whole batch. OF is a displacement map. This way you are linearly going from whatever interval to a bounded interval but you are lossing the scale information. If your batch is big enough max and min value will be approximately the same for each batch and this scale info won’t be lost.

The best you can do is checking some paper about it.

5

thank a lot! Have a nice day!