Hello all,
i couldn’t really find any good resource on what is the best way to stack my 3D tensor data into a batch which i can feed into my model.forward() .
Currently what i got from the tutorials is a
std::vector<torch::jit::IValue> inputs;
When i push_back my tensors into the vector and pass it directly into the forward function it interprets the number of elements as input arguments which is not what i want.
How can i concat my tensors into a given batch size?
I also found torch::cat() but i need a dynamic way to stack the data tensors into batches.
Created an account just to answer this question after finding no answer and despairing. Specifically to answer the question of “how to submit different sized tensors as forward input”. Original poster seems to have wanted this to implement a batch with different sized samples within each batch which… is a whole extra can of worms. I’ll be focusing on how to implement forward(self, dynamicListOfInputs).
As best I can tell you simply cannot. The methods which others suggest such as
std::vectortorch::jit::IValue inputs;
Assume that the forward function itself (e.g. the python code if transferring to C++ like me) is defined as forward(self, x, y, z).
However if you tried to keep things neat and tidy like me and defined your function as forward(self, listOfInputs) then you’ll get no end of errors. Libtorch does not like lists of undefined size.
You only option in libtorch seem to be:
- use cat() to make one large super-tensor and break it up in the forward function
- if tensors are different sizes and cat() is no an option, explicitly pass each one in separately as forward(x,y,z).
The tensors themselves can be dynamically sized (e.g. to have different batch sizes based on available GPU) however the number of tensors in the list cannot. You must explicitly list each tensor as a separate argument in the definition of the forward function.
Hopefully this helps someone out there.
I am new to using libtorch, but if you have you tensors in something like,
std::vector<torch::Tensor> tensors;
then this should work well:
torch::stack(tensors, 0);
Better yet, you could load your inputs into a std::vector<torch::Tensor> wrapped custom dataset class inheriting from torch::data::Dataset<> followed by using the torch::data::transforms::Stack<>() transformation through the map function in Dataset class, which I think (not sure of it yet) will lazily apply the torch::stack operation on batches of input in your training loop. You can find a good demo for MNIST images here.