Custom DataLoader

I saw the tutorial on custom dataloader. However, the class function has loading data functions too. I have tensors pair images, labels. How can I convert them into DataLoader format without using CustomDataset class??

@yf225. Please help me to resolve this issue.

I am getting this error:

error: invalid cast to abstract class type ‘CustomDataset’
auto data = CustomDataset(train_tensor_indices, train_tensor_lab).map(torch::data::transforms::Stack<>());

note: because the following virtual functions are pure within ‘CustomDataset’:
class CustomDataset : public torch::data::Dataset
In file included from /libtorch/include/torch/csrc/api/include/torch/data/datasets.h:3:0,
from /libtorch/include/torch/csrc/api/include/torch/data.h:4,
from /libtorch/include/torch/csrc/api/include/torch/all.h:4,
from /libtorch/include/torch/csrc/api/include/torch/torch.h:3,

Custom Dataset class:

class CustomDataset : public torch::data::Dataset {

    torch::Tensor states_, labels_;

	CustomDataset(torch::Tensor emb_indices, torch::Tensor labels) {
		states_ = emb_indices;
     	labels = labels;
     torch::data::Example<> get(size_t index) override {
     	torch::Tensor embedding_index = states_[index];
     	torch::Tensor Label = labels_[index];
     	return {embedding_index.clone(), Label.clone()};
     }; };
auto data = CustomDataset(train_tensor_indices, train_tensor_lab).map(torch::data::transforms::Stack<>());

I think you need to implement a size() as well. See here:
Dataset inherited from BatchDataset and it has a pure virtual function size() which is not overridden.

As the comment said, if you don’t need it, just return an empty c10::optional

Thank you for the suggestion. I will try and let you know.

@glaringlee Thanks for the suggestion. It worked. I need one more clarification.though not in scope to this question. The loss functions doesn’t support Long Tensors for backward pass?

I need more details on ‘loss’ question.
Do you have any log or error msg? I think we should support Long Tensors (the tensor’s datatype is Long correct? correct me if I am wrong)


for(auto& batch : trainloader) {
auto inputs = at::_cast_Long(;
auto targets = at::_cast_Long(;

            totalsize = totalsize + inputs.size(0);
            auto output = Net.forward(inputs);
            auto loss = torch::nn::functional::cross_entropy(inputs, targets);

Error Message:
terminate called after throwing an instance of ‘c10::Error’
what(): “log_softmax_lastdim_kernel_impl” not implemented for ‘Long’ (operator() at aten/src/ATen/native/cpu/SoftMaxKernel.cpp.AVX2.cpp:238)

frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0x6a (0x7f6e629dcaaa in /libtorch/lib/

frame #1: + 0x154c8ff (0x7f6e54fe58ff in /libtorch/lib/

frame #2: at::native::log_softmax_cpu(at::Tensor const&, long, bool) + 0x54a (0x7f6e548ecc1a in /libtorch/lib/

frame #3: + 0x115ef07 (0x7f6e54bf7f07 in /libtorch/lib/

frame #4: + 0x11b16ab (0x7f6e54c4a6ab in /libtorch/lib/

frame #5: + 0x2d497a2 (0x7f6e567e27a2 in /libtorch/lib/

frame #6: + 0x11b16ab (0x7f6e54c4a6ab in /libtorch/lib/

frame #7: at::native::log_softmax(at::Tensor const&, long, c10::optionalc10::ScalarType) + 0x160 (0x7f6e548ea0e0 in /libtorch/lib/

frame #8: + 0x1227333 (0x7f6e54cc0333 in /libtorch/lib/

frame #9: + 0x2ce3cf5 (0x7f6e5677ccf5 in /libtorch/lib/

frame #10: + 0x12987cc (0x7f6e54d317cc in /libtorch/lib/

yes, data is long type.

I see. log_softmax_lastdim_kernel_impl is implemented for floating types. Specifically, it supports double, float and torch::Half only. You might need to use double instead of long.

@glaringlee. But Embedding layer only takes Long Tensors as input. I have to use Embedding layers. Is there any work around to avoid this issue? Should I raise an issue in the official github repo?

Can you post more code?
Based on the code piece you post here, I think you can cast your inputs and targets to double first. If you have other layers that need long, you can cast them to long later.


struct RandCNN : nn::Module {
RandCNN(int32_t &vocabsize, int32_t &embeddim, int32_t &maxlen, int32_t &numclasses)

  : embed(nn::Embedding(nn::EmbeddingOptions(vocabsize, embeddim))),

    conv1(nn::Conv1d(nn::Conv1dOptions(embeddim, kNumfilters, kFilterSizes[0])

    conv2(nn::Conv1d(nn::Conv1dOptions(embeddim, kNumfilters, kFilterSizes[1])

    conv3(nn::Conv1d(nn::Conv1dOptions(embeddim, kNumfilters, kFilterSizes[2])

    pool1(nn::MaxPool1d(nn::MaxPool1dOptions(maxlen - kFilterSizes[0] +1)

    pool2(nn::MaxPool1d(nn::MaxPool1dOptions(maxlen - kFilterSizes[1] +1)

    pool3(nn::MaxPool1d(nn::MaxPool1dOptions(maxlen - kFilterSizes[2] +1)

    fc(nn::Linear(nn::LinearOptions(kNumfilters * 3, numclasses))),

    drop(nn::Dropout(nn::DropoutOptions().p(kDropValue))) {
// register_module() is needed if we want to use the parameters() method later on
register_module("embed", embed);
register_module("conv1", conv1);
register_module("conv2", conv2);
register_module("conv3", conv3);
register_module("poo11", pool1);
register_module("pool2", pool2);
register_module("pool3", pool3);
register_module("fc", fc);


torch::Tensor forward(torch::Tensor x) {
x = embed(x);
x = at::transpose(x, 1, 2);
torch::Tensor out1, out2, out3, out;
out1 = torch::relu(conv1(x));
out2 = torch::relu(conv2(x));
out3 = torch::relu(conv3(x));
out1 = pool1(out1);
out2 = pool2(out2);
out3 = pool3(out3);
out = at::_cat({out1, out2, out3}, 1);
out = at::_unsafe_view(out, {at::size(out, 0), at::size(out, 1)});
out = drop(out);
out = fc(out);
//out = torch::log_softmax(out, /*dim=*/1);

return out;

My dataformat will be a dataloader with Indices and Labels. The first layer of model Embedding converts them to embedding vectors. So, inputs will be Long Tensors.

I see.
In your forward function, when calling the log_softmax, you can do the conversion
out = torch::log_softmax( if you need Long type as output

okay so i will use long type as input to embedding layer. then type cast to double after this so that it will be backward compatible? But the problem is back-propogation thorugh long tensors is giving error.

A typo in my previous reply, it should be out = torch::log_softmax(, 1).to(torch::kLong)

It should be backward compatible. Can I see your back-propogation error log?
The previous error log you showed me is caused by calling log_softmax in forward.

@glaringlee I have identified the issue after careful debugging. In this line I should be passing outputs and targets to calculate loss however I was passing inputs. However, the error log is not useful to trace back the error. It should have reported shapes mismatch instead was reporting about log_softmax.

Thank you for all the patience and time in helping me throughout.

Great! I think the reason log_softmax complain is that your input is Long type :slight_smile: .

I guess there should have been a better error log. Because clearly it is a shape mismatch.