Returning more than 2 tensors in libtorch

Hello all ,
I am writing a simple dataloader for an object detection task and trying to return 3 tensors from the get method. However I intended to return a tuple containing all three but I am unable to do so as the return type is torch::data::Example. Could anyone please guide me as to how I can return all three tensors?
TIA

Wouldn’t returning all 3 tensors work with torch::data::Example as given e.g. in the get method of your dataset:

torch::data::Example<> get(size_t i) {
    return {torch::ones(i), torch::ones(i), torch::ones(i)};
  }

I did try that but it threw an error saying no matching constructor found and I was only able to return at most two tensors.
Anyway I will try that once again. Thanks a lot for replying ptrbk.

I just stumbled across this exact same issue when trying to create a custom dataset that returns 3 tensors vs. 2. I’m not crazy well versed in C++ but it looks like @ptrblck’s solution wouldn’t work (and didn’t for me at least) according to the source code. I think we’re limited to 1 or 2 outputs based on the templates in that header. I’d be interested to hear of any potential workarounds or solutions to this issue.

Did you try to write a custom Example class with more than two return types?
The linked class shows:

template <typename Data = Tensor, typename Target = Tensor>

so I would assume you can reuse this template and add more types to it?

Wow that was a quick response! I’m actually working on that approach now. I’ll see how it goes and report back with what I find.

I’m mostly a Python dev though so I’m a bit out of my element here.

EDIT: It looks like writing a custom Example class and providing it as a template argument when creating a custom dataset subclass (per this line here) should work. I’ll write up a minimal example here when I figure it out.

1 Like

Apologies in advance if this doesn’t compile directly on a copy/paste. I wrote this based loosely on my actual implementation but haven’t compiled or tested this exact bit of code. I also haven’t fully tested this approach in a training loop but the approach compiles and can be used to do batched inference with my TorchScript model. Note that I had to create a custom Example and custom Stack class to accommodate the extra model input in transforms as well.

I’m not great with C++ but I’m wondering if there’s some way that parameter packs/variadic templating can be used to remove the 2 input limit on custom datasets? I started probing that with the Example class but don’t have any strong ideas on how to carry that forward into the transforms or anything.

It looks like example.data and example.target are used quite often in examples, transforms, etc. vs. taking the indexing/unpacking approach to getting inputs from a batch in Python. Maybe it’s worth exploring the Python approach here as well.

#include <tuple>
#include <vector>
#include <fliesystem>

#include <torch/torch.h>

template <typename Data = torch::Tensor, typename Target = torch::Tensor,
          typename Mask = torch::Tensor>
struct Example {
  using DataType = Data;
  using TargetType = Target;
  using MaskType = Mask;

  Data data;
  Target target;
  Mask mask;

  Example() = default;
  Example(Data data, Target target, Mask mask)
      : data(std::move(data)), target(std::move(target)),
        mask(std::move(mask)) {}
}

template <typename ExampleType>
struct Stack : public torch::data::transforms::Collation<ExampleType> {
  ExampleType apply_batch(std::vector<ExampleType> examples) override {
    std::vector<torch::Tensor> xs, ys, masks;
    xs.reserve(examples.size());
    ys.reserve(examples.size());
    masks.reserve(examples.size());
    for (auto &example : examples) {
      xs.push_back(std::move(example.data));
      ys.push_back(std::move(example.target));
      masks.push_back(std::move(std::get<0>(example.rest)));
    }
    return {torch::stack(xs), torch::stack(ys), torch::stack(masks)};
  }
};

using MyExample = Example<torch::Tensor, torch::Tensor, torch::Tensor>;
class MyDataset : public torch::data::datasets::Dataset<MyDataset, MyExample> {
private:
  std::vector<torch::Tensor> xs, ys, masks;

public:
  MyDataset(const std::filesystem::path dataFpath) {
    xs, ys, masks = loadDataFromFile(dataFpath);
  }
  virtual ~MyDataset();
  MyExample get(size_t index) override {
    torch::Tensor x = xs[index];
    torch::Tensor y = ys[index];
    torch::Tensor mask = masks[index];

    return {x, y, mask};
  }
  torch::optional<size_t> size() const override { return xs.size(); }
}

auto main() -> int {
  const std::filesystem::path dataDir = "/path/to/data/";
  const int batchSize = 32;
  auto dataset = MyDataset(dataDir).map(Stack<MyExample>());
  auto dataLoader =
      torch::data::make_data_loader<torch::data::samplers::SequentialSampler>(
          std::move(dataset), batchSize);
}
1 Like

Thanks for sharing your approach!
I’m sure the libtorch interface could be polished a bit more, but I also think that training in C++/libtorch is still an edge case (compared to inference) which is why the training API might not have gotten a lot attention.

I agree on both fronts. There aren’t many usecases where training in C++ makes more sense than training in Python and converting a model to TorchScript for inference in C++.

I’m working on model training in C++ for an iOS app ultimately. Unfortunately I’ll need to be able to do on device training for my usecase so I’m certain I’ll come across a lot of unpolished edges as I move forward. I’ll be sure to open issues and make PRs where I can along the way.

Thanks for the help on this!

1 Like

Hello krzim,
I am facing a similar problem, where my network has two branches, each awaiting for a different size of tensor. I could not succeed in making your code example work. Didi you end up with a solution ? Would you mind sharing it ?
For my problem, I was also wondering if my Example could contain a pair of tensors as the data and another tensor as the target. But, here too, I could not obtain in a working code. I am not easy enough with C++ for that. Any idea welcome.
Thanks

Just to share my solution, I ended up with this simpler code:

auto makeDataPrototypes()
{
  std::vector<std::tuple<torch::Tensor, torch::Tensor, torch::Tensor>> dataset;
  for (int i = 0; i < 100; i++)
    dataset.push_back(std::make_tuple(torch::randn({2, 41, 41}), torch::randn({2}), torch::randn({1})));
  return dataset;
}

struct Example {
  torch::Tensor input1;
  torch::Tensor input2;
  torch::Tensor output;

  Example() = default;
  Example(torch::Tensor input1, torch::Tensor input2, torch::Tensor output)
      : input1(std::move(input1)), input2(std::move(input2)), output(std::move(output)) {}
};

struct Stack : public torch::data::transforms::Collation<Example> {
  Example apply_batch(std::vector<Example> examples) override {
    std::vector<torch::Tensor> inputs1, inputs2, outputs;
    inputs1.reserve(examples.size());
    inputs2.reserve(examples.size());
    outputs.reserve(examples.size());
    for (auto &example : examples) {
      inputs1.push_back(std::move(example.input1));
      inputs2.push_back(std::move(example.input2));
      outputs.push_back(std::move(example.output));
    }
    return {torch::stack(inputs1), torch::stack(inputs2), torch::stack(outputs)};
  }
};

class MyDataset : public torch::data::datasets::Dataset<MyDataset, Example> {

  private:
    std::vector<std::tuple<torch::Tensor, torch::Tensor, torch::Tensor>> dataset_;
  
  public:
    MyDataset() :
      dataset_(makeDataPrototypes()) {
        std::cout << "nb proto = " << dataset_.size() << '\n';
    }
    Example get(size_t index) override {
      auto tuple = dataset_[index];
      return {std::get<0>(tuple), std::get<1>(tuple), std::get<2>(tuple)};
    }
    torch::optional<size_t> size() const override { 
      return dataset_.size(); 
    }
};

int main(int argc, char *argv[]) {
  
  auto dataset = MyDataset().map(Stack());
  std::cout << "nb proto in main = " << dataset.size().value() << '\n';
  auto loader = torch::data::make_data_loader<torch::data::samplers::RandomSampler>(std::move(dataset), 10);
  std::cout << "nb proto after loader creation = " << dataset.size().value() << '\n';
 /* for (auto& batch : *loader) 
    auto outputs = net->forward(batch.input1, batch.input2);*/
}

The only thing that is still mysterious for me is that nb proto in main after loader creation is 0:

nb proto = 100
nb proto in main = 100
nb proto after loader creation = 0

It is due to the std::move, but is it required? Anyway the loop on batch examples seems to work fine.

1 Like

With regard to the implementation of torch::Tensor (use of an intrusive_ptr), std::move is not necessary because it breaks with the externality of the reference counter. (the semantic move is used internally in the base class)

As stated in the TensorBody.h documentation

Tensor is a “generic” object holding a pointer to the underlying TensorImpl object, which has an embedded reference count. In this way, Tensor is similar to boost::intrusive_ptr.

For example:

void func(Tensor a) {
   Tensor b = a;
   ...
}

In this example, when we say Tensor b = a, we are creating a new object that points to the same underlying TensorImpl, and bumps its reference count. When b goes out of scope, the destructor decrements the reference count by calling release() on the TensorImpl it points to. The existing constructors, operator overloads, etc. take care to implement the correct semantics.

It is therefore possible that by using std::move in your Example constructor, you lose the underlaying construction of your Tensor’s.

Not sure at 100%, I do not use the move semantic with the libtorch API.

1 Like

If you’re finding that the other options aren’t working for you, you might consider defining a custom return type for your dataset. This method seems more aligned with the intended usage of LibTorch’s dataset functionalities.

Firstly, create a structure for your custom return type and then use this type with torch::data::datasets::Dataset. In this context, the first template parameter is the type of your custom dataset class (MyCustomDataset), and the second parameter is the type of the data item your dataset will return (MyCustomExample). By default, the data item type for a dataset in LibTorch is torch::data::Dataset::Example<at::Tensor, at::Tensor>, but you can customize it as shown below:

struct MyCustomExample {
    torch::Tensor image;
    torch::Tensor mask;
    std::string classname;
};

class MyCustomDataset : public torch::data::datasets::Dataset<MyCustomDataset, MyCustomExample> {
  public:
    // ...
    MyCustomExample get(size_t index) override;
    // ...
};