Why I get a model negative loss returned

I’m quite new in python and pytorch.

I have perhaps simple problem.
I get negative loss numbers returned.

This is the output:

Epoch [1/100], Step [10/63], Loss: 0.1118
Epoch [1/100], Step [20/63], Loss: -0.3783
Epoch [1/100], Step [30/63], Loss: -1.4037
Epoch [1/100], Step [40/63], Loss: -3.8306
Epoch [1/100], Step [50/63], Loss: -7.8438
Epoch [1/100], Step [60/63], Loss: -14.6053
Epoch [2/100], Step [10/63], Loss: -27.3804
Epoch [2/100], Step [20/63], Loss: -36.3176
Epoch [2/100], Step [30/63], Loss: -41.2981

My datareader from CSV file:

class CSV_Data(Dataset):
    def __init__(self, path):
        xy = np.loadtxt("data/"+path, delimiter=",", dtype=np.float32, skiprows=1, usecols=(0,1,2,3))
        self.x = torch.tensor(xy[:, 0:2], requires_grad=True)
        self.y = torch.from_numpy(xy[:, 2:])
        self.n_samples = xy.shape[0]
    def __getitem__(self, index):
        sample = self.x[index], self.y[index]
        return sample

    def __len__(self):
        # len(dataset)
        return self.n_samples

My model:

class RegressionReluNN(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(RegressionReluNN, self).__init__()
        self.linear1 = nn.Linear(in_features=input_size, out_features=hidden_size)
        self.relu = nn.ELU()
        # Or maybe Relu ???
        self.linear2 = nn.Linear(in_features=hidden_size, out_features=2)

    def forward(self, x):
        out = self.linear1(x)
        out = self.relu(out)
        out = self.linear2(out)
        # sigmoid at the end ???
        y_predict = torch.sigmoid(out)
        return y_predict

Training loop:

# criterion = nn.CrossEntropyLoss()
criterion = nn.MultiLabelSoftMarginLoss()

n_total_steps = len(train_loader) # This is for test

for epoch in range(num_epochs):
    for i, (open_close, min_max) in enumerate(train_loader):
        # Forward
        outputs = model(open_close)
        # print(outputs)
        # sys.exit()
        # min_max_target = torch.argmax(min_max, 1)

        # print(min_max_target)
        # sys.exit()

        loss = criterion(outputs, min_max)

        # Backward

        if (i+1) % 10 == 0:
            print(f'Epoch [{epoch+1}/{num_epochs}], Step [{i+1}/{n_total_steps}], Loss: {loss.item():.4f}')

Data I have for the input and target is:

[[float32, float32]]
[[float32, float32]]

I also tied to use CrossEntropuLoss but but got confused with conversion, the actual: min_max_target = torch.argmax(min_max, 1) didn’t get me any positive results.

Hi Anton!

You are probably passing target values that are greater than one into
your MultiLabelSoftMarginLoss loss criterion.

min_max is read in from some unspecified data file, and then used as
the target for MultiLabelSoftMarginLoss, which expects its target
to consist of values that are either 0.0 or 1.0. If you pass it values
that are greater than 1.0, it can return a negative loss.


K. Frank

Thank you for your answer, it rally helped me to understand my mistake.