Reduce loss function to get max-valued solution instead of min-valued solution

My model is used to predict values based on an minimising a loss function L. But, the loss function doesn’t have a single global minima value, but rather a large number of places where it achieves global minima.

So, the model is based like this:

Model Input is [nXn] tensor (let’s say: inp=[ [i_11, i_12, i_13, ..., i_1n],[i_21, i_22, ..., i_2n],...,[i_n1,i_n2, ..., i_nn] ]) and model output is [nX1] tensor (let’s say: out1=[o_1, o_2,..., o_n ])

Output tensor is out1 is passed in a function f to get out2 (let’s say: f(o_1, o_2, o_3,..., o_n)=[O_1, O_2, O_3, ..., O_n] )

These 2 values (i.e., out1 and out2) are minimised using MSELoss i.e., Loss = ||out1 - out2||

Now, there are a lot of values for [o_1, o_2, ..., o_n] for which the Loss goes to minimum.

But, I want the values of [o_1, o_2, ..., o_n] for which |o_1| + |o_2| + |o_3| + ... + |o_n| is maximum

Right now, the weights are initialised randomly:

self.weight = torch.nn.parameter.Parameter(torch.FloatTensor(in_features, out_features)) for some value of in_features and out_features

But by doing this, I am getting the values of [o_1, o_2, ..., o_n] for which |o_1| + |o_2| + |o_3| + ... + |o_n| is minimum.

I know this problem can be solved by without using deep-learning, but I am trying to get the results like this for some task computation.

Is there a way to change this to get the largest values predicted at the output of the neural net?

Or is there any other technique (backpropagation change) to change it to get the desired largest valued output?

Thanks in advance.


Based on the answer, out1=[o_1, o_2,..., o_n ] is tending to zero-valued tensor. In the initial epochs, out2=[O_1, O_2, O_3, ..., O_n] takes very large values, but subsequently comes down to lower values.

A snippet of code below will give the idea:

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import numpy as np

class Model(nn.Module):
    def __init__(self, inp_l, hid_l, out_l=1):
        super(Model, self).__init__()
        self.lay1 = nn.Linear(inp_l ,hid_l)
        self.lay2 = nn.Linear(hid_l ,out_l)
        self.dp = nn.Dropout(p=0.5)
    def forward(self, inp):
        self.out1= torch.tensor([]).float()
        for row in range(x.shape[0]):
            y = self.lay1(inp[row])
            y = F.relu(y)
            y = self.dp(y.float())
            y = self.lay2(y)
            y = F.relu(y)
            self.out1=, y))
        return self.out1.view(inp.shape[0],-1)

def function_f(inp, out1):
    Some functional computation is done to return out2.
    return out2

def train_model(epoch):
    t = time.time()
    out1 = model(inp)
    out2 = function_f(inp, out1)
    loss1 = ((out1-out2)**2).mean()
    loss2 = -out1.abs().mean()
    loss_train = loss1 + loss2

    if epoch%40==0:
        print('Epoch: {:04d}'.format(epoch+1),
                  'loss_train: {:.4f}'.format(loss_train.item()),
                  'time: {:.4f}s'.format(time.time() - t))

model= Model(inp_l=10, hid_l=5, out_l=1)
optimizer = optim.Adam(model.parameters(), lr=0.001)

inp = torch.randint(100, (10, 10))

for ep in range(100):

But, out1 value goes to trivial solution i.e., zero-valued tensor which is the minimum valued solution. As mentioned before EDIT, I want to get the max-valued solution.

Thank you.