[C++] [beginner] why does tensor cast not work ? / RuntimeError: expected scalar type Byte but found Float

C++
1

Hi,

I’m pretty new to pytorch and even more to C++ frontend.
I’m currently trying to use yolov5 on C++ and keep having type problems no matter how hard I try to cast it. I Do it once in the options of the tensor, and then after having copied data from the cv::mat .
I also tried to change the cv::mat conversion type to others types (such as CV_8UC3 for example) without success.

I tried this solution (you can see it in my code) without success : Type conversion in Libtorch for C++

Here is (the relevant part of) my code :

//___________________________________________________________________________________________
//___________________________________________UTILS___________________________________________
//___________________________________________________________________________________________

bool load_preproc_detect(cv::Mat& src , cv::Mat& dst ,  std::string& path , float scale ){

  //___________________read_image_data
  src = cv::imread( path );
  if (src.data==NULL){
    std::cerr << "empty image file, maybe invalid path?" << std::endl;
  }

  //________detect_preprocess
  cv::cvtColor( src , src , cv::COLOR_RGB2BGR );//opencv uses bgr
  src.convertTo(dst ,  CV_32FC3, 1.0 / 255, 0); //normalization and type matching tensor input type

  //arbitrary resize to reduce computation speed(watch for the precision tradeoff)
  cv::Size dst_sz = src.size();
  dst_sz.height = (int) dst_sz.height * scale;
  dst_sz.width = (int) dst_sz.width * scale;
  cv::resize(dst , dst ,  dst_sz , 1 ,  1);
}









//___________________________________________________________________________________________
//___________________________________________MAIN____________________________________________
//___________________________________________________________________________________________
int main(int argc, char* argv[])
{
  //_______clear screen__________
  std::cout <<"oui"<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl<<std::endl;


  //______________defining path
  std::string root_path = "/home/kubler/sdsp-rea/";
  std::string detect_weight_path = root_path+"model/detect/yolo_test.torchscript.pt" ;
  std::string data_path = root_path + "temp_data/RGB_0001.png";


  //______________________CUDA SETUP_______________________
  torch::Device device(torch::kCPU);


  //___________________read_image_data
  cv::Mat img ; //= cv::imread(data_path);
  cv::Mat detect_img;
  cv::Mat pose_img;
  float scale = 1/2.5;
    load_preproc_detect(img , detect_img , data_path , scale );
  std::cout << "data preprocessed" << std::endl;

  //_________________TORCH SETUP

  //load torch model
  torch::jit::script::Module detect_model , pose_model;
  detect_model = load_from_torchscript(detect_weight_path , device);

  //set to inference only mode : grads are all frozen for safety
  c10::InferenceMode guard;

  //input_tensor_preparation
  auto detect_options =   torch::TensorOptions()
  .dtype(at::kByte) // trying to force Byte type
  .layout(torch::kStrided)
  .device(device)
  .requires_grad(false);

  torch::Tensor detect_input;// = torch::empty({detect_sz.width , detect_sz.height} , detect_options );
  detect_input = torch::from_blob(detect_img.data, {1 ,  detect_img.rows, detect_img.cols , detect_img.channels()} , detect_options);
  // 2 different ways of casting i have seen here and not working : i get the same output 
  //detect_input = detect_input.item().to<Byte>();
  detect_input = detect_input.to(torch::kByte);
  detect_input = detect_input.permute({0, 3 , 1 , 2});

  std::vector<torch::jit::IValue> detect_dataset;
  detect_input = detect_input.to(device);
  detect_dataset.push_back(detect_input);

  //______________________INFERENCE_______________________

  torch::Tensor detect_output = detect_model.forward(detect_dataset).toTensor();

  std::cout << "end" << std::endl;
  return 0;
}

and here is the output

data preprocessed
model loaded
terminate called after throwing an instance of 'std::runtime_error'
  what():  The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/models/yolo.py", line 33, in forward
    _22 = getattr(self.model, "2")
    _23 = getattr(self.model, "1")
    _24 = (getattr(self.model, "0")).forward(x, )
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    _25 = (_22).forward((_23).forward(_24, ), )
    _26 = (_20).forward((_21).forward(_25, ), )
  File "code/__torch__/models/common.py", line 19, in forward
    _8 = torch.slice(_7, 3, 1, 9223372036854775807, 2)
    input = torch.cat([_2, _4, _6, _8], 1)
    return (_0).forward(input, )
            ~~~~~~~~~~~ <--- HERE
class Conv(Module):
  __parameters__ = []
  File "code/__torch__/models/common.py", line 29, in forward
  def forward(self: __torch__.models.common.Conv,
    input: Tensor) -> Tensor:
    _9 = (self.act).forward((self.conv).forward(input, ), )
                             ~~~~~~~~~~~~~~~~~~ <--- HERE
    return _9
class C3(Module):
  File "code/__torch__/torch/nn/modules/conv.py", line 11, in forward
    input: Tensor) -> Tensor:
    _0 = self.bias
    x = torch._convolution(input, self.weight, _0, [1, 1], [1, 1], [1, 1], False, [0, 0], 1, False, False, True, True)
        ~~~~~~~~~~~~~~~~~~ <--- HERE
    return x

Traceback of TorchScript, original code (most recent call last):
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/conv.py(440): _conv_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/conv.py(443): forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/data/det_track/yolov5/models/common.py(48): forward_fuse
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/data/det_track/yolov5/models/common.py(206): forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/data/det_track/yolov5/models/yolo.py(155): forward_once
/home/kubler/data/det_track/yolov5/models/yolo.py(123): forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1039): _slow_forward
/home/kubler/.local/lib/python3.6/site-packages/torch/nn/modules/module.py(1051): _call_impl
/home/kubler/.local/lib/python3.6/site-packages/torch/jit/_trace.py(959): trace_module
/home/kubler/.local/lib/python3.6/site-packages/torch/jit/_trace.py(744): trace
export.py(35): export_torchscript
export.py(154): run
export.py(187): main
export.py(192): <module>
RuntimeError: expected scalar type Byte but found Float

I am a bit confused because in the original yolov5 code, the input value type is float as you can see just under but the error i keep having is that the model expect to receive a float type value.

for path, img, im0s, vid_cap in dataset:
    img = torch.from_numpy(img).to(device)
    img = img.half() if half else img.float()  # uint8 to fp16/32
    img = img / 255.0  # 0 - 255 to 0.0 - 1.0
    if len(img.shape) == 3:
        img = img[None]  # expand for batch dim

I hope I have been precise enough in my explanations, thank you for your help \o

2

Based on the error message and your code it seems as if you are trying to use a ByteTensor in a common model, which uses float32 by default.
You would have to make sure the input dtype is matching the model parameters, so transform the input to float32 and it should work (I don’t think many operations support ByteTensors).

3

Yesss! it worked thank you!

Can I ask where I can find an explanation to it?
because on this type table the only references of ByteTensor are for integers.

Also the code error was

RuntimeError: expected scalar type Byte but found Float

I know that I have at the beginnng a float cv::Mat of dtype CV_32FC3 that I converted to torch::tensor of dtype at::kByte. So does the at::kByte conversion somehow keeps the float type without letting the model using it ?

I am not contesting your answer just trying to understand how this error was formed because it displayed the opposite of what I understood I had to do.

Thank you again !

4

The error message might be a bit confusing, as PyTorch assumes that the first argument to an operation has the desired dtype (which is a ByteTensor in torch._convolution(input, self.weight, ...)) and complains about the self.weight, which is a FloatTensor.

5

Ok I might have understood. Is this a feature or a bug ? :grimacing:

Anyway thank you for your precious help !