Accessors versus TensorIterators

kazem · May 25, 2023, 3:36am

What is the difference between Accessors and TensorIterators? And what is the use case of each?
I have read about TensorItreators in these:

github.com

kurtamohler/pytorch-TensorIterator-examples/blob/main/examples.cpp

#include <iostream>
#include <cassert>
#include <ATen/ATen.h>
#include <ATen/native/TensorIterator.h>
#include <ATen/native/cpu/Loops.h>
#include <ATen/native/ReduceOpsUtils.h>
#include <ATen/AccumulateType.h>

void example1() {
  at::Tensor a = at::ones({10});
  at::Tensor b = at::ones({10});
  at::Tensor out = at::zeros({0});
  std::cout
    << "\n==========\n"
    << "example1:"
    << std::endl;

  //======== Start blog post code =========
  at::TensorIteratorConfig iter_config;
  iter_config

This file has been truncated. show original

github.com

pytorch/pytorch/blob/main/aten/src/ATen/TensorIterator.h

#pragma once

#include <ATen/TensorMeta.h>
#include <ATen/core/Dimname.h>
#include <ATen/core/Range.h>
#include <ATen/core/TensorBase.h>
#include <c10/core/DynamicCast.h>
#include <c10/util/FunctionRef.h>
#include <c10/util/MaybeOwned.h>
#include <c10/util/SmallVector.h>
#include <c10/util/TypeCast.h>
#include <c10/util/irange.h>

#include <array>
#include <bitset>

C10_CLANG_DIAGNOSTIC_PUSH()
#if C10_CLANG_HAS_WARNING("-Wshorten-64-to-32")
C10_CLANG_DIAGNOSTIC_IGNORE("-Wshorten-64-to-32")
#endif

This file has been truncated. show original

And about accessors and packed accessors towards the end here:
https://pytorch.org/tutorials/advanced/cpp_extension.html

Basically, if I want to loop over a tensor elements and carry out a simple computation which one should I use? does the use case of (Accessors(CPU Tensors)/PackAccesors (CUDA Tensors) VS TensorIterators) depend on the device of the tensor?

ptrblck · May 25, 2023, 3:51am

The TensorIterator can be used to execute e.g. elementwise kernels as seen in the first link while accessors are used to index data inside a kernel from the tensor instead of using the pointer with strides and indices.

kazem · May 25, 2023, 4:58am

Is there any preference over which one better or faster?! Is the speed of using an accessor the same? Also When one should use TensorIterator over a cuda tensor instead of defining a cuda kernel from scratch?

ptrblck · May 25, 2023, 5:07am

These objects are used for different use cases as already mentioned.
If you want to write an elementwise kernel you could use the TensorIterator and allow it to iterate all elements of your tensor. On the other hand, if you want to index a tensor manually and apply any operation the accessor can be used.
I don’t think you would see a huge difference between using the accessor vs. manually indexing the pointer.

kazem · May 25, 2023, 5:20am

specifically, I would like to reimplement scipy.ndimage.find_objects using pytorch, only on the cpu at the moment, since it is the basic block for cellular analysis in biomedical imaging: Extract the cells from the image using an already built mask and then extract measurements from them.

They use a Numpy Iterator object in the base C code to iterate over all entries of the numpy array and record the smallest and largest index of each object in each direction/dimension in a numpy array pointer called regions, while knowing how many objects there is. The base C code can be found here:

github.com

scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c

/* Copyright (C) 2003-2005 Peter J. Verveer
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above
 *    copyright notice, this list of conditions and the following
 *    disclaimer in the documentation and/or other materials provided
 *    with the distribution.
 *
 * 3. The name of the author may not be used to endorse or promote
 *    products derived from this software without specific prior
 *    written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
 * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

This file has been truncated. show original

And its numpy iterator struct is found here:

github.com

scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h

/* Copyright (C) 2003-2005 Peter J. Verveer
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above
 *    copyright notice, this list of conditions and the following
 *    disclaimer in the documentation and/or other materials provided
 *    with the distribution.
 *
 * 3. The name of the author may not be used to endorse or promote
 *    products derived from this software without specific prior
 *    written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
 * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

This file has been truncated. show original

Do you recommend implementing it using an Accessor or a TensorIterator?
Also if I write the code in torch C++ API the best I can, Can you please help me correct/debug/improve it, and add it as a new function to torch C++ API?
I would really appreciate it since it will help me build a new torch api for biomedical imaging applications.

kazem · June 5, 2023, 4:35pm

@ptrblck I already started a thread that is slowly coming to life:

github.com/pytorch/pytorch

scipy.ndimage.find_objects

opened 08:32PM - 24 May 23 UTC

kazemSafari

feature module: nn triaged needs research

### 🚀 The feature, motivation and pitch This function a basic building block …of any biomedical image analysis application. It gives a list of tuple of slices of coordinates of labelled objects/cells within a mask image of dtype Uint16 or Uint8, assuming the image background is 0, and the labelled objects go from 1, 2, ..., max_label. I was wondering it is possible to implement it in torch C++ using a simple TensorIterator. Basically the simplest case would be it takes a 2D tensor of size (H, W) as input and outputs a tensor of slices of size (N, 2) where N is the number of objects, and each row is [slice(start,end,step), slice(start,end,step)]. ### Alternatives The implementation in C numpy can be found here: https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_measure.c which uses Iterators defined here: https://github.com/scipy/scipy/blob/v1.10.1/scipy/ndimage/src/ni_support.h ### Additional context Can it also be extended to allow extract objects from a tensor of dimension (B, C, W, H) where B is the batch size, C the number of channels and W is the width and H is the height. cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki