Example model won't learn


(Filipel Filipel) #1

I have problem with porting Python example https://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-custom-nn-modules to C++

#include "stdafx.h"



struct NeuralNetworkImpl : torch::nn::Module
{
	torch::nn::Linear linear_1, linear_2;



	NeuralNetworkImpl(uint32_t input, uint32_t hidden, uint32_t output) :
	linear_1(torch::nn::LinearOptions(input, hidden)),
	linear_2(torch::nn::LinearOptions(hidden, output))
	{}

	torch::Tensor forward(torch::Tensor x)
	{
		x = linear_1->forward(x);
		x = x.clamp(0);
		x = linear_2->forward(x);
		return x;
	}
};
TORCH_MODULE(NeuralNetwork);



int main()
{
	uint32_t batchSize = 64;
	uint32_t input = 1000;
	uint32_t hidden = 100;
	uint32_t output = 10;

	torch::Tensor x = torch::randn({ batchSize, input });
	torch::Tensor y = torch::randn({ batchSize, output });

	NeuralNetwork neuralNetwork = NeuralNetwork(input, hidden, output);
	torch::optim::SGD optimizer(neuralNetwork->parameters(), torch::optim::SGDOptions(0.01));

	for (uint32_t i = 1; i <= 500; ++i)
	{
		torch::Tensor yPredicted = neuralNetwork->forward(x);
		torch::Tensor loss = torch::mse_loss(yPredicted, y, 2);

		std::cout << i << "   " << loss.item() << std::endl;

		optimizer.zero_grad();
		loss.backward();
		optimizer.step();
	}

	std::cin.ignore();
    return 0;
}

Compiles, and run but ‘loss’ is not changing at all during 500 epochs. I assume error in my implementation, but I cannot find it.


(Martin Huber) #2

You forgot to register your submodules. Do the following, replace this line

linear_1(torch::nn::LinearOptions(input, hidden)),
linear_2(torch::nn::LinearOptions(hidden, output))

by

linear_1(register_module("linear_1", torch::nn::Linear(input, hidden))),
linear_2(register_module("linear_2", torch::nn::Linear(hidden, output)))

So nn::Module will know of the layers.