How to copy layer's weight to cv::Mat in C++

C++
1

Hello everyone,

I am trying to copy convolutional layer weights to a std::vectorcv::Mat structure using this function:

void tensor2mat(torch::Tensor &t, std::vector<cv::Mat> &out)
{
	caffe2::TypeMeta dtype;
	dtype = t.dtype();
	c10::ScalarType type;
	type = t.scalar_type();

	std::cout << "Tensor Format: {dims} ( depth - type)" << std::endl;
	std::cout << t.sizes() << " ( " << dtype << " - " << type << " )" << std::endl;


	int mattype = CV_8U;
	int typesize;
	if (dtype == torch::kInt64 || dtype == torch::kFloat64)
	{
		typesize = sizeof(int64);
		mattype = CV_64F;
	}
	else if (dtype == torch::kFloat32)
	{
		typesize = sizeof(float);
		mattype = CV_32F;
	}
	else if (dtype == torch::kByte || dtype == torch::kChar)
	{
		typesize = sizeof(char);
		mattype = CV_8U;
	}


	if (t.dim() == 4)
	{
		// if number of dimensions is 4
		// NBatch x NChannels x H x W

		out.clear();
                out.resize(t.size(0));

		for (size_t i = 0; i < out.size(); i++) {
			std::vector<cv::Mat> channels(t.size(1));
			for (size_t j = 0; j < t.size(1); j++)
			{
				// Iterate through channels
				torch::Tensor one_plane_tensor = t[i][j];
                                // one_plane_tensor shape is [ 11, 11 ]
				std::cout << one_plane_tensor.sizes() << " depth: " << one_plane_tensor.dtype() << std::endl;
				one_plane_tensor.to(torch::kCPU);
                                
                                // one_mat is CV_32FC1
				cv::Mat one_mat(one_plane_tensor.size(0), one_plane_tensor.size(1), CV_MAKETYPE(mattype,1));

                                // here I get an error
				std::memcpy((void *)one_mat.data, one_plane_tensor.data_ptr(), typesize * one_plane_tensor.numel());
				channels[j] = one_mat;
			}
			cv::merge(channels, out[i]);
		}
		
		// Create a grid with displaying all the images in the vector
		cv::Mat output = ext::draw::composeLayerImage(std::pair < std::string, std::vector<cv::Mat> >("prueba", imgs), true);


	}
}

Layer’s weight tensor shape is [64, 3, 11, 11] so the output vector should contain 64 11x11x3 images, so that I can display the content of the weights in a window.

ComposeLayer is just a function to visualize all the images in a grid.

I call the function in this way


cv::Mat prueba;

// tensor2mat(net->conv1.get()->weight, prueba);
tensor2mat(net->conv1.get()->named_parameters()["weight"], prueba);

When I call it, an error is produced in memcpy instruction. Could anyone give me a hint as to why this is happening?

Thank you very much.

2

Answering myself, I have found that the problem came because the tensor one_plane_tensor was on GPU and not on CPU. This can be solve just by assigning the output of ‘to’ function to another tensor that will be on CPU.

torch::Tensor tensor_in_cpu = one_plane_tensor.to(torch::kCPU);

and it works.