Very simple categorisation problem

5

If, for some reason, you’d like to frame the problem to be useable for CrossEntropyLoss, here is one way you might go about it:

import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np


outputExcepted = torch.tensor([[42, 4300, 55000]])
data = torch.tensor([[0, 0, 0]], dtype=torch.float32)

class FCNet(nn.Module):
    def __init__(self):
        super(FCNet, self).__init__()
        self.layer1 = nn.Linear(3, 55001*3)

    def forward(self, x):
        x = torch.relu(self.layer1(x))
        return x.view(3,55001)


model = FCNet()
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)
num_epochs = 10000

model.train()

for epoch in range(num_epochs):
    optimizer.zero_grad()
    outputs = model(data)
    loss = criterion(outputs, outputExcepted.view(-1))
    loss.backward()
    optimizer.step()
    print('Epoch [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, loss.item()))

print("Predictions", torch.argmax(outputs, dim=1))

When it is finished, you can check that the predictions are correct and match your outputExpected values.

It’s going to be hit and miss, though, depending on how the initial weights and biases get instantiated. This is because the model depth is so thin and the only inputs are zeroes. You could try playing around with the learning rate or try initializing your biases to 1s in your init with self.layer1.bias = nn.Parameter(torch.ones_like(self.layer1.bias)).

1 Like