Efficient conversion of multiple cv::Mat images into a single torch::Tensor

C++
1

:question: Questions and Help

I have an image that I slice into multiple equal segments and then put into a vector. So essentially I have a vector of cv::Mat objects. I want to take these multiple image segments and convert them into a single tensor in the most efficient way.

Current approach

std::vector<cv::Mat> imageSegments;
// ...
// code to populate imageSegments vector, say we have a 100 of these per image
auto imageSegTensor = torch::empty({100, 3, 50, 30 }, torch::kFloat); // each image seg is 50 x 30 x 3
auto imageSegTensorInterface = imageSegTensor.accessor<float, 4>();

int segCount = 0;
for (const auto & seg : imageSegments) {
    for (int i = 0; i < seg.rows; ++i) {
        for (int j = 0; j < seg.cols; ++j) {
            cv::Vec3b pixel = seg.at<cv::Vec3b>(i, j)
            imageSegTensorInterface[segCount][0][i][j] = pixel.val[2];
            imageSegTensorInterface[segCount][1][i][j] = pixel.val[1];
            imageSegTensorInterface[segCount][2][i][j] = pixel.val[0];
        }
    }
    segCount++
}

As you can see, this is copying the data from the cv::Mat to the tensor, and there is obviously a way to make this more efficient. I tried to follow #14330 but I could not figure out how to extend this to multiple images in a single tensor.

How would one go about conjuring a solution that allows for the transfer of multiple image segments/images of type cv::Mat to a single torch::Tensor by just using data pointers for instance?

2 Likes
2

you can reference here :

auto img_tensor = torch::CUDA(torch::kFloat32).tensorFromBlob(img_float.data, {1, 224, 224, 3});
img_tensor = img_tensor.permute({0, 3, 1, 2});

https://github.com/pytorch/pytorch/issues/14219

1 Like
3

Have you found a more efficient way to do this yet? I reckon cuixing158_1’s way is just turning a single cv::Mat image to a single Tensor, which is not what you were looking for.

4

Given @cuixing158_1 solution, one could convert a std::vector<cv::Mat> into a std::vector<torch::Tensor>. Now, we could use torch::cat, to convert the std::vector<torch::Tensor> into a single tensor. For example

std::vector<torch::Tensor> vec;
torch::Tensor ten = torch::cat(vec);