# Multi-task regression problem using Pytorch (Problem: Same output for all test data)

I am trying to multi-task regression problem.

Input shape: 200 * 60000, Output shape: 200*3 (here, 200 = Total number of data, 60000 = number of features)

So, for each data point, I have to predict 3 values (continuous).

Sample Code:

``````class Classifier(nn.Module):
def __init__(self,input_nodes):
super(Classifier, self).__init__()
self.input_nodes = input_nodes

self.sharedlayer = nn.Sequential(
nn.Linear(input_nodes, 300),
nn.ReLU(),
nn.Dropout(),
nn.Linear(300, 100),
nn.ReLU(),
nn.Dropout(),
)

self.att1 = nn.Sequential(
nn.Linear(100, 40),
nn.ReLU(),
nn.Dropout(),
nn.Linear(40, 20),
nn.ReLU(),
nn.Dropout(),
nn.Linear(20, 1)
)
self.att2 = nn.Sequential(
nn.Linear(100, 40),
nn.ReLU(),
nn.Dropout(),
nn.Linear(40, 20),
nn.ReLU(),
nn.Dropout(),
nn.Linear(20, 1)
)
self.att3 = nn.Sequential(
nn.Linear(100, 40),
nn.ReLU(),
nn.Dropout(),
nn.Linear(40, 20),
nn.ReLU(),
nn.Dropout(),
nn.Linear(20, 1)
)

def forward(self, x):

h_shared = self.sharedlayer(x)
out1 = self.att1(h_shared)
out2 = self.att2(h_shared)
out3 = self.att3(h_shared)

return out1, out2, out3

criterion = nn.MSELoss()
for epoch in range(n_epochs):
running_loss = 0
i = 0
model.train()
i = i + 1
out1, out2, out3 = model(data)

l1 = criterion(out1, label[:,0].view(-1,1))
l2 = criterion(out2, label[:,1].view(-1,1))
l3 = criterion(out3, label[:,2].view(-1,1))

loss = (l1 + l2 + l3)
loss.backward()
optimizer.step()
``````

Problem: The model always produces the same value for all test data.

Example: Suppose, 3 Test data:

For output1: 3.5 3.5 3.5
For output2: 9.5 9.5 9.5
For output3: 0.2 0.2 0.2

Why does it generate the same value for all test data?

Could you check the `bias` values of the last linear layers?
Sometimes bad hyperparameters (e.g. high learning rate) throws the parameters out so that only the bias will be predicted (often the mean value of your target).

Bias value is different than the output values.

For 11 test data —
Generated Outputs for output3 are:

``````tensor([[0.8806],
[0.8806],
[0.8806],
[0.8806],
[0.8806],
[0.8806],
[0.8806],
[0.8806],
[0.8806],
[0.8806],
``````

Access the weights and bias using the below code:

``````a) model.att3[6].weight
Parameter containing:
tensor([[-0.0960,  0.1035, -0.0493,  0.4335, -0.2495,  0.1452,  0.1481, -0.4376,
-0.0052,  0.0901,  0.0533, -0.0907,  0.0778,  0.1182, -0.4527,  0.2129,