Libtorch C++ convert a Tensor to cv:Mat (single channel

Hi, big masters and huge Gods,

I wanna ask a question hope anyone open this link can give me a help which is convert a Tensor to cv::Mat.

I have a Tensor outputs from my model and it’s [1, 19, 32, 43] shape, which is, batch_size, 19 instances, and 32x43 is the featuremap size.

I have it’s type: CPUFloatType. And I want converts it into cv::Mat, here is what I do:

 cv::Mat one_mat(heatMapsTensor.size(2), heatMapsTensor.size(3), CV_32FC1);
std::memcpy((void *) one_mat.data, one_heat_map.data_ptr(), sizeof(torch::kFloat32) * one_heat_map.numel());

heatMapsTensor is the tensor described above. one_mat is what I need to got.

But I got the mat incorrect. what did I wrong?

I suspect 2 maybe mistake:

  • the type maybe not float32, but I do not know which it is;
  • the mat shape is not wrong

Any body could help me out?

Since you are using the tensor.data_ptr() you would not use ...sizeof(torch::kFloat32) * numel... but rather ...sizeof(float) * numel...

Try something like:

std::memcpy(cv_pointer, tensor.data_ptr(), tensor.numel() * sizeof(float));

Assuming you are using floats.
You may also want to have a look (Convert torch::tensor to cv::Mat . from @mhubii )

1 Like

thanks for you reply!!

I changed into this:

	  cv::Mat one_mat(heatMapsTensor.size(1), heatMapsTensor.size(2), CV_32FC1);
	  std::memcpy(one_mat.data, one_heat_map.data_ptr(), sizeof(float) * one_heat_map.numel());

This time, the cv::Mat values seems normal, but result still not right.

what I want , is convert a tensor in multi dimension: [19, 32, 46] into a vector, length is 19, and every member in vetor is cv::Mat with size of 32,46.

Now, dims is right, the issue is the value converted into cv::Mat is always not right, it is not the same value in tensor.

Ahh,
then those are two kinds of problems.
What you can do is loop over the first dimension and slice the tensor and then do std::mmcpy

So something like this

std::vector<cv::Mat> list_of_mat;
for (int i=0; i < tensor.size(0); i++) {
// tensor slicing
// std::memcpy to dummy
list_of_mat.pushback(dummy)
}
1 Like

This is actually what I am doing for now:

std::vector<cv::Mat> heatMaps(heatMapsTensor.size(0));

	for (size_t i = 0; i < heatMaps.size(); i++) {
	  torch::Tensor one_heat_map = heatMapsTensor[i];
	  cv::Mat one_mat(heatMapsTensor.size(1), heatMapsTensor.size(2), CV_32FC1);
	  std::memcpy(one_mat.data, one_heat_map.data<float>(), sizeof(float) * one_heat_map.numel());
	  heatMaps[i] = one_mat;
	}

I just saw this usage tensor_a.data<float>() which seems a clue so I changed to this but still not right.

I am trying another thread:

get the tensor pointer say float* p, then copy it’s data to cv::Mat. However, how should I get the point of a tensor ? using tensor_a.data_ptr()? But that is (void*) not what I need float*

From cv2 to TORCH:

 cv::cvtColor(frame, frame, CV_BGR2RGB);
 frame.convertTo(frame, CV_32FC3, 1.0f / 255.0f);
 auto input_tensor = torch::from_blob(frame.data, {1, frame_h, frame_w, kCHANNELS});
 input_tensor = input_tensor.permute({0, 3, 1, 2});

From TORCh to cv2:

input_tensor = input_tensor.to(at::kCUDA);
                torch::Tensor out_tensor = module->forward({input_tensor}).toTensor();

                out_tensor = out_tensor.squeeze().detach().permute({1, 2, 0});
                out_tensor = out_tensor.mul(255).clamp(0, 255).to(torch::kU8);
                out_tensor = out_tensor.to(torch::kCPU);
                cv::Mat resultImg(frame_h, frame_w, CV_8UC3);
                std::memcpy((void *) resultImg.data, out_tensor.data_ptr(), sizeof(torch::kU8) * out_tensor.numel());
2 Likes

this really works!!!

A million thanks to you!
Would it be possible to also post how to get CV_32C3 format instead of CV_8UC3?
I cant get it working, im getting artifacts