Efficient conversion of multiple cv::Mat images into a single torch::Tensor


#1

:question: Questions and Help

I have an image that I slice into multiple equal segments and then put into a vector. So essentially I have a vector of cv::Mat objects. I want to take these multiple image segments and convert them into a single tensor in the most efficient way.

Current approach

std::vector<cv::Mat> imageSegments;
// ...
// code to populate imageSegments vector, say we have a 100 of these per image
auto imageSegTensor = torch::empty({100, 3, 50, 30 }, torch::kFloat); // each image seg is 50 x 30 x 3
auto imageSegTensorInterface = imageSegTensor.accessor<float, 4>();

int segCount = 0;
for (const auto & seg : imageSegments) {
    for (int i = 0; i < seg.rows; ++i) {
        for (int j = 0; j < seg.cols; ++j) {
            cv::Vec3b pixel = seg.at<cv::Vec3b>(i, j)
            imageSegTensorInterface[segCount][0][i][j] = pixel.val[2];
            imageSegTensorInterface[segCount][1][i][j] = pixel.val[1];
            imageSegTensorInterface[segCount][2][i][j] = pixel.val[0];
        }
    }
    segCount++
}

As you can see, this is copying the data from the cv::Mat to the tensor, and there is obviously a way to make this more efficient. I tried to follow #14330 but I could not figure out how to extend this to multiple images in a single tensor.

How would one go about conjuring a solution that allows for the transfer of multiple image segments/images of type cv::Mat to a single torch::Tensor by just using data pointers for instance?


(cuixing) #2

you can reference here :

auto img_tensor = torch::CUDA(torch::kFloat32).tensorFromBlob(img_float.data, {1, 224, 224, 3});
img_tensor = img_tensor.permute({0, 3, 1, 2});

https://github.com/pytorch/pytorch/issues/14219