torch::nn::CrossEntropyLoss does no work

Hi , I was traying to translate this example:
pytorchTutorial/11_softmax_and_crossentropy.py at master · python-engineer/pytorchTutorial · GitHub , lines 57-70
from “Python-PyTorch” to “C++ - PyTorch”

TEST01

# target is of size nSamples = 1
# each element has class label: 0, 1, or 2
# Y (=target) contains class labels, not one-hot
Y = torch.tensor([0])

# input is of size nSamples x nClasses = 1 x 3
# y_pred (=input) must be raw, unnormalizes scores (logits) for each class, not softmax
Y_pred_good = torch.tensor([[2.0, 1.0, 0.1]])
Y_pred_bad = torch.tensor([[0.5, 2.0, 0.3]])
l1 = loss(Y_pred_good, Y)
l2 = loss(Y_pred_bad, Y)

print(f'PyTorch Loss1: {l1.item():.4f}')
print(f'PyTorch Loss2: {l2.item():.4f}')

My translation C++

torch::nn::CrossEntropyLoss criterion;

	auto Y = torch::tensor({0}, torch::kFloat32);

	auto Y_good = torch::tensor({ {2.0, 1.0, 0.1} }, torch::kFloat32);
	auto Y_bad = torch::tensor({ {0.5, 2.0, 0.3} }, torch::kFloat32);

	auto l1 = criterion(Y_good, Y);//<-------------ERROR????

	std::cout << l1 << std::endl;

But every time, it gave me an error

Unhandled exception at 0x00007FF80AE7D759 in Test01.exe: Microsoft C++ exception: c10::Error at memory location 0x000000E17614A6F0.

It is possible that there is an error, because I didn’t use any “forward” method. So I create a SECOND test with a linear regression that use “forward” to test this possibility. (I base my second try using this tutorial https:/ / github . com/prabhuomkar/pytorch-cpp/blob/master/tutorials/popular/blitz/training_a_classifier/src/main.cpp), because making things harder always solve the problem :smile:

TEST02

class “MyNetImpl” ---- mynnet.h

#pragma once
#include <torch/torch.h>
class MyNetImpl : public torch::nn::Module 
{
public:
    MyNetImpl();
    torch::Tensor forward(torch::Tensor x);
private:
    torch::nn::Linear line1;    
};
TORCH_MODULE(MyNet);

with mynnet.cpp

#include "mynnet.h"

MyNetImpl::MyNetImpl():
	line1(torch::nn::LinearOptions(9, 2))
{
	register_module("line1", line1);
}

torch::Tensor MyNetImpl::forward(torch::Tensor x)
{
	auto out = line1->forward(x);
	return out;	
}

in the main, I use the dataset from (Index of /ml/machine-learning-databases/breast-cancer-wisconsin), it has:

Attribute ||----- Domain


  1. Sample code number ||----- id number
  2. Clump Thickness ||----- 1 - 10
  3. Uniformity of Cell Size ||-----1 - 10
  4. Uniformity of Cell Shape ||-----1 - 10
  5. Marginal Adhesion ||-----1 - 10
  6. Single Epithelial Cell Size ||-----1 - 10
  7. Bare Nuclei ||-----1 - 10
  8. Bland Chromatin ||-----1 - 10
  9. Normal Nucleoli ||-----1 - 10
  10. Mitoses ||-----1 - 10
  11. Class: ||-----(2 for benign, 4 for malignant)

Important:
Using torch::nn::BCELoss works well, however y want to use torch::nn::CrossEntropyLoss, for that reason the 11.Class is loaded like these:

benign
{0,1}

malignant
{1,0}

with that I can use: torch::nn::LinearOptions(9, 2)

the method LoadereSCV return a pair of tensors, that are in this format:

auto targets = torch::empty({ size_, 2 }, torch::kFloat32);
auto input = torch::empty({ size_, 9 }, torch::kFloat32);
void Test02()
{
	std::string path = "//breast-cancer-wisconsin.data";

	int dataloaderNum = 699;
	int num_epochs = 100;
	float learning_rate = 0.01;
	torch::DeviceType device_type;

	if (torch::cuda::is_available()) {
		std::cout << "CUDA available! Training on GPU." << std::endl;
		device_type = torch::kCUDA;
	}
	else {
		std::cout << "Training on CPU." << std::endl;
		device_type = torch::kCPU;
	}


	auto train_dataset = LoadereSCV(path, 0)
		.map(torch::data::transforms::Stack<>());
	auto train_loader = torch::data::make_data_loader<torch::data::samplers::RandomSampler>(
		std::move(train_dataset), dataloaderNum);


	MyNet net = MyNet();

	net->to(device_type);

	// // Define a Loss function and optimizer
	torch::nn::CrossEntropyLoss criterion;// (torch::nn::CrossEntropyLossOptions().ignore_index(-100).reduction(torch::kMean));

	torch::optim::SGD optimizer(net->parameters(), torch::optim::SGDOptions(0.001).momentum(0.9));
	//torch::optim::Adam optimizer(net->parameters(), torch::optim::AdamOptions());


	// Train the network
	for (size_t epoch = 0; epoch < num_epochs; ++epoch)
	{
		double running_loss = 0.0;

		int i = 0;
		for (auto& batch : *train_loader)
		{
			// get the inputs; data is a list of [inputs, labels]
			auto inputs = batch.data.to(device_type);
			auto labels = batch.target.to(device_type);

			// zero the parameter gradients
			optimizer.zero_grad();

			// forward + backward + optimize
			auto outputs = net->forward(inputs);
			std::cout << "inputs" << std::endl;
			std::cout << inputs << std::endl;
			std::cout << "labels" << std::endl;
			std::cout << labels << std::endl;
			std::cout << "outputs" << std::endl;
			std::cout << outputs << std::endl;
			
			
            auto loss = criterion(outputs, labels);//<---ERROR-------

			loss.backward();
			optimizer.step();

			// print statistics
			running_loss += loss.item<double>();
			if (i % 10 == 0)
			{
				std::cout << "[" << epoch + 1 << ", " << i << "] loss: "
					<< running_loss << '\n';
				running_loss = 0.0;
			}
			i++;
		}
		std::cout << "Epoch = " << epoch << " i=" << i - 1 << " \n" << std::endl;
	}
	std::cout << "Finished Training\n\n";
}

int main()
{
Test02();
system("pause");
return 0;
}

These are my:

inputs
 0.5000  0.6000  0.6000  0.8000  0.6000  1.0000  0.4000  1.0000  0.4000
 0.3000  0.1000  0.4000  0.1000  0.2000  0.1000  0.1000  0.1000  0.1000
 0.7000  0.5000  0.3000  0.7000  0.4000  1.0000  0.7000  0.5000  0.5000
 0.1000  0.1000  0.1000  0.1000  0.2000  0.2000  0.1000  0.1000  0.1000
 0.5000  0.2000  0.2000  0.2000  0.1000  0.1000  0.2000  0.1000  0.1000
.
.
.
.
 0.1000  0.1000  0.1000  0.1000  0.1000  0.1000  0.2000  0.1000  0.1000
 1.0000  0.6000  0.6000  0.2000  0.4000  1.0000  0.9000  0.7000  0.1000
 0.2000  0.3000  0.1000  0.1000  0.5000  0.1000  0.1000  0.1000  0.1000
 0.6000  1.0000  1.0000  1.0000  0.8000  1.0000  1.0000  1.0000  0.7000
 0.8000  1.0000  1.0000  1.0000  0.8000  1.0000  1.0000  0.7000  0.3000
[ CUDAFloatType{699,9} ]
labels
 0  1
 1  0
 0  1
 0  1
 0  1
.
.
.
 0  1
 0  1
 0  1
 0  1
 1  0
[ CUDAFloatType{699,2} ]
outputs
-0.0505  0.2318
 0.0214  0.1086
-0.0315  0.2268
-0.0436  0.1762
 0.0056  0.2524
.
.
.
-0.0477  0.2313
-0.0516  0.2239
-0.0434  0.2346
 0.0050  0.2397
 0.2576  0.1424
[ CUDAFloatType{699,2} ]

Well that’s it, could somebody help me with this. In these 2 tests torch::nn::CrossEntropyLoss does not work, however if you use
github . com /prabhuomkar / pytorch-cpp/blob/master/tutorials/popular/blitz/training_a_classifier/src/main.cpp
tutorial. It works fine. Even with my custom dataset of image 36x36.

I find the ERROR in test02, the problem appears when I created the labels. I send

targets.to(torch::kFloat32)

but it should be:

targets.to(torch::kInt64)

after that change everything works great, for test02