# Training loss is not decreasing below a specific value

**URL:** <https://discuss.pytorch.org/t/training-loss-is-not-decreasing-below-a-specific-value/78687>\
**Category:** vision\
**Created:** [April 27, 2020, 4:57pm UTC](https://discuss.pytorch.org/t/training-loss-is-not-decreasing-below-a-specific-value/78687 "2020-04-27T16:57:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Harsh\_Chaudhary](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/harsh_chaudhary/32/20207_2.png) [@Harsh\_Chaudhary](https://discuss.pytorch.org/u/Harsh_Chaudhary)\
**Post date:** [April 27, 2020, 4:57pm UTC](https://discuss.pytorch.org/t/training-loss-is-not-decreasing-below-a-specific-value/78687/1 "2020-04-27T16:57:48Z")

</div>

I am trying to solve a problem that I found in deep learning with pytorch course on Udacity: **“Predict whether a student will get selected or rejected by the university”**.

But i am getting the training loss ~0.2000 every time. The loss is not falling below this value. Could you please go through the code to give me feedback?

```auto

import pandas as pd
import numpy as np
data = pd.read_csv('student_data.csv')

import matplotlib.pyplot as plt

def plot_points(data):
    X = np.array(data[['gre', 'gpa']])
    Y = np.array(data['admit'])
    admitted = X[np.argwhere(Y==1)]
    rejected = X[np.argwhere(Y==0)]
    plt.scatter([s[0][0] for s in rejected], [s[0][1] for s in rejected], color='red')
    plt.scatter([s[0][0] for s in admitted], [s[0][1] for s in admitted], color='cyan')
    plt.xlabel('Test (GRE)')
    plt.ylabel('Grades (GPA)')

plot_points(data)

data = data.drop('rank', axis=1)

# Normalizing the values
data['gre'] /= max(data['gre'])
data['gpa'] /= max(data['gpa'])

# splitting in train and test set
sample = np.random.choice(data.index, size=int(len(data)*0.9), replace=False)
train_data, test_data = data.iloc[sample], data.drop(sample)

train_features = train_data.drop('admit', axis=1)
train_labels = train_data['admit']

import torch
from torch import nn, optim
import torch.nn.functional as F 

# converting dataFrames into tensors
input_tensor = torch.from_numpy(train_features.values).type(torch.FloatTensor)
label_tensor = torch.from_numpy(train_labels.values).type(torch.FloatTensor)

class Net(nn.Module):
    def __init__ (self):
        super(Net, self). __init__ ()
        
        self.fc1 = nn.Linear(2, 4)
        self.fc2 = nn.Linear(4, 1)
   
    def forward(self, x):
        
        x = F.relu(self.fc1(x))
        x = (self.fc2(x))
        return x

model = Net()

criteron = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.003)

losses=[]
EPOCHS = 501
for e in range(EPOCHS):
    train_loss = 0
    
    output = model(input_tensor)
    loss = criteron(output, label_tensor.view(360, 1))
    
    train_loss += loss.item()
    losses.append(train_loss)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if e%100==0:
        print('epoch {}: Training loss: {:.4f}'.format(e+1, train_loss))

plt.plot(losses)  

```

[here](https://github.com/udacity/deep-learning-v2-pytorch/blob/master/keras/student-admissions-keras/student_data.csv) is how the data looks like.

---

<div class="post-metadata">

**Author:** ![abodh\_ltd](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abodh_ltd/32/22637_2.png) [@abodh\_ltd](https://discuss.pytorch.org/u/abodh_ltd)\
**Post date:** [April 27, 2020, 11:37pm UTC](https://discuss.pytorch.org/t/training-loss-is-not-decreasing-below-a-specific-value/78687/2 "2020-04-27T23:37:18Z")

</div>

Just some random suggestions:

1. can you put `optimizer.zero_grad()` before `loss = criteron(output, label_tensor.view(360, 1))`.
2. set `model.train()` before you start the training process

let me know if this does not work.

---

<div class="post-metadata">

**Author:** ![Harsh\_Chaudhary](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/harsh_chaudhary/32/20207_2.png) [@Harsh\_Chaudhary](https://discuss.pytorch.org/u/Harsh_Chaudhary)\
**Post date:** [April 28, 2020, 3:51am UTC](https://discuss.pytorch.org/t/training-loss-is-not-decreasing-below-a-specific-value/78687/3 "2020-04-28T03:51:07Z")

</div>

thank you for your reply mate!

But this won’t help because optimizer.zero\_grad() is required before the gradient calculation i.e loss.backward(). So it wont make any difference.

setting model.train() also wont help because models are by default into train mode until we explicitly put it into evaluation mode.