Multi dimensional slicing issue

C++
1

I am trying to perform a multi dimensional slice operation on a Tensor to obtain a ROI using the following code:

// Image resolution is 800x600
cv::Mat img = cv::imread("test.png");
auto img_tensor = torch::from_blob(img.data, {img.cols, img.rows, 3}, torch::kByte);
auto slice_tensor = img_tensor.index({
	torch::indexing::Slice(0, img.cols/2),
	torch::indexing::Slice(0, img.rows/2),
	torch::indexing::Slice()
}).contiguous(); // contiguous is required to copy the raw data to a Mat and write it to a png.

cv::Mat image_as_mat(cv::Size(slice_tensor.size(0),slice_tensor.size(1)),CV_8UC3, slice_tensor.data_ptr());
cv::imwrite("slice_img.png", image_as_mat);

From what I can tell, this is correct, but the result is not as expected:

pt_forum
pt_forum1201×600 4.96 KB

If the slice is set to the same size as the image, the output image is exactly the same. Which I assume is due to optimisations skipping slicing code. If the slice is changed to:

auto slice_tensor = img_tensor.index({
	torch::indexing::Slice(0, img.cols-1),
	torch::indexing::Slice(0, img.rows),
	torch::indexing::Slice()
}).contiguous();

The output image is corrupted like the output slice in the image above (but a different pattern).

Interestingly if I try slicing a smaller amount of data, it works as expected:

auto test = torch::arange(3*6*6).reshape(3, 6, 6});
auto slice = test.index({
	torch::indexing::Slice(),
	torch::indexing::Slice(0, 3),
	torch::indexing::Slice(0, 3)
}).contiguous();
std::cout << "test: " << test << std::endl;
std::cout << "slice: " << slice << std::endl;

Outputs:

test: (1,.,.) =
   0   1   2   3   4   5
   6   7   8   9  10  11
  12  13  14  15  16  17
  18  19  20  21  22  23
  24  25  26  27  28  29
  30  31  32  33  34  35

(2,.,.) =
  36  37  38  39  40  41
  42  43  44  45  46  47
  48  49  50  51  52  53
  54  55  56  57  58  59
  60  61  62  63  64  65
  66  67  68  69  70  71

(3,.,.) =
   72   73   74   75   76   77
   78   79   80   81   82   83
   84   85   86   87   88   89
   90   91   92   93   94   95
   96   97   98   99  100  101
  102  103  104  105  106  107
[ CPULongType{3,6,6} ]
slice: (1,.,.) =
   0   1   2
   6   7   8
  12  13  14

(2,.,.) =
  36  37  38
  42  43  44
  48  49  50

(3,.,.) =
  72  73  74
  78  79  80
  84  85  86
[ CPULongType{3,3,3} ]

Am I missing something? I’m really at a loss as to what the issue might be.

Environment:
libtorch: v1.6 CPU (downloaded pre built binaries, but also built locally for a debug build)
os: windows 10 x64
compiler: msvc v16.7.5 x64

2

The output indicates an issue between the slicing where a permutation might be needed, but I’m not sure which method causes it.
Could you run the same test you’ve used for the tensor slicing operation on the OpenCV arrays and check, which operation interleaves the data?

3

Hi @ptrblck, thanks for your help!

I’m not entirely sure what you are asking, but I did some more testing and I found that if I use a square image tensor (800x800), I get the correct (uncorrupted) output slice image, regardless of what kind of slice I take (square or not). So its some kind of issue with the structure of the underlying data. Is this related to what you are referring to?

4

Yes, more or less.
I think this shape and thus also the slice might be wrong:

{img.cols, img.rows, 3}

OpenCV should use a [height, width, channels] layout, so shouldn’t it be {img.rows, img.cols, 3}?

5

Yep, that was exactly the issue!

It appeared to work when the tensor was not modified as for both the input and output the dimensions were reversed.

Thanks @ptrblck!