Libtorch C++ convert a Tensor to cv:Mat (single channel

(Jin Tian) #1

Hi, big masters and huge Gods,

I wanna ask a question hope anyone open this link can give me a help which is convert a Tensor to cv::Mat.

I have a Tensor outputs from my model and it’s [1, 19, 32, 43] shape, which is, batch_size, 19 instances, and 32x43 is the featuremap size.

I have it’s type: CPUFloatType. And I want converts it into cv::Mat, here is what I do:

 cv::Mat one_mat(heatMapsTensor.size(2), heatMapsTensor.size(3), CV_32FC1);
std::memcpy((void *) one_mat.data, one_heat_map.data_ptr(), sizeof(torch::kFloat32) * one_heat_map.numel());

heatMapsTensor is the tensor described above. one_mat is what I need to got.

But I got the mat incorrect. what did I wrong?

I suspect 2 maybe mistake:

  • the type maybe not float32, but I do not know which it is;
  • the mat shape is not wrong

Any body could help me out?

(Lucas Müller) #2

Since you are using the tensor.data_ptr() you would not use ...sizeof(torch::kFloat32) * numel... but rather ...sizeof(float) * numel...

Try something like:

std::memcpy(cv_pointer, tensor.data_ptr(), tensor.numel() * sizeof(float));

Assuming you are using floats.
You may also want to have a look (Convert torch::tensor to cv::Mat . from @mhubii )

1 Like
(Jin Tian) #3

thanks for you reply!!

I changed into this:

	  cv::Mat one_mat(heatMapsTensor.size(1), heatMapsTensor.size(2), CV_32FC1);
	  std::memcpy(one_mat.data, one_heat_map.data_ptr(), sizeof(float) * one_heat_map.numel());

This time, the cv::Mat values seems normal, but result still not right.

what I want , is convert a tensor in multi dimension: [19, 32, 46] into a vector, length is 19, and every member in vetor is cv::Mat with size of 32,46.

Now, dims is right, the issue is the value converted into cv::Mat is always not right, it is not the same value in tensor.

(Lucas Müller) #4

Ahh,
then those are two kinds of problems.
What you can do is loop over the first dimension and slice the tensor and then do std::mmcpy

So something like this

std::vector<cv::Mat> list_of_mat;
for (int i=0; i < tensor.size(0); i++) {
// tensor slicing
// std::memcpy to dummy
list_of_mat.pushback(dummy)
}
1 Like
(Jin Tian) #5

This is actually what I am doing for now:

std::vector<cv::Mat> heatMaps(heatMapsTensor.size(0));

	for (size_t i = 0; i < heatMaps.size(); i++) {
	  torch::Tensor one_heat_map = heatMapsTensor[i];
	  cv::Mat one_mat(heatMapsTensor.size(1), heatMapsTensor.size(2), CV_32FC1);
	  std::memcpy(one_mat.data, one_heat_map.data<float>(), sizeof(float) * one_heat_map.numel());
	  heatMaps[i] = one_mat;
	}

I just saw this usage tensor_a.data<float>() which seems a clue so I changed to this but still not right.

I am trying another thread:

get the tensor pointer say float* p, then copy it’s data to cv::Mat. However, how should I get the point of a tensor ? using tensor_a.data_ptr()? But that is (void*) not what I need float*

(dambo) #6

From cv2 to TORCH:

 cv::cvtColor(frame, frame, CV_BGR2RGB);
 frame.convertTo(frame, CV_32FC3, 1.0f / 255.0f);
 auto input_tensor = torch::from_blob(frame.data, {1, frame_h, frame_w, kCHANNELS});
 input_tensor = input_tensor.permute({0, 3, 1, 2});

From TORCh to cv2:

input_tensor = input_tensor.to(at::kCUDA);
                torch::Tensor out_tensor = module->forward({input_tensor}).toTensor();

                out_tensor = out_tensor.squeeze().detach().permute({1, 2, 0});
                out_tensor = out_tensor.mul(255).clamp(0, 255).to(torch::kU8);
                out_tensor = out_tensor.to(torch::kCPU);
                cv::Mat resultImg(frame_h, frame_w, CV_8UC3);
                std::memcpy((void *) resultImg.data, out_tensor.data_ptr(), sizeof(torch::kU8) * out_tensor.numel());
1 Like
How to load pictures in C++ with Libtorch?
How to covert torch::Tensor to cv::Mat
(Jin Tian) #7

this really works!!!