# Simple ANN model not training

**URL:** <https://discuss.pytorch.org/t/simple-ann-model-not-training/136434>\
**Category:** ignite\
**Created:** [November 10, 2021, 12:26am UTC](https://discuss.pytorch.org/t/simple-ann-model-not-training/136434 "2021-11-10T00:26:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![paragG](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/ebca7d/32.png) [@paragG](https://discuss.pytorch.org/u/paragG)\
**Post date:** [November 10, 2021, 12:26am UTC](https://discuss.pytorch.org/t/simple-ann-model-not-training/136434/1 "2021-11-10T00:26:12Z")

</div>

Hi, I am new to PyTorch. I am trying to train a simple ANN model for regression. The issue is my loss doesn’t seem to be changing much over ~10 epochs. The exact same model/loss/optimizer, on the same data shows very different loss in another implementation. The two implementations primarily differ in data prep. The data is in table format - each row has a target col and a bunch of feature cols. I would greatly appreciate any help (been stuck for several days). Below is my code:  
First I read in the data in a pandas dataframes df and df\_test. Next, I split df into train\_data+val\_data using stratified split. The rest of the code is below:

 ![image](https://discuss.pytorch.org/uploads/default/original/3X/4/c/4cc804f85706b724659a3c7977a093c6bc30ea78.png)

The val\_loss changes between 3.84-3.85 over 10 epochs. The other implementation starts with a val\_loss \< 2 and by 6 epochs is \< 1 (same batch size as mine). Sorry for the lengthy post. I would appreciate any help (may be an obvious mistake I am overlooking?). Thanks.

---

<div class="post-metadata">

**Author:** ![my3bikaht](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/50afbb/32.png) [@my3bikaht](https://discuss.pytorch.org/u/my3bikaht)\
**Post date:** [November 10, 2021, 7:01am UTC](https://discuss.pytorch.org/t/simple-ann-model-not-training/136434/2 "2021-11-10T07:01:14Z")

</div>

it’s hard to read on screenshot  
you can use google colab to share your notebook

---

<div class="post-metadata">

**Author:** ![vfdev-5](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vfdev-5/32/3535_2.png) [@vfdev-5](https://discuss.pytorch.org/u/vfdev-5)\
**Post date:** [November 10, 2021, 8:41am UTC](https://discuss.pytorch.org/t/simple-ann-model-not-training/136434/3 "2021-11-10T08:41:43Z")

</div>

@paragG yes, an executable code would be nice to have.  
Do you see that the model is training, e.g. training batch loss is decreasing ?

---

<div class="post-metadata">

**Author:** ![paragG](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/ebca7d/32.png) [@paragG](https://discuss.pytorch.org/u/paragG)\
**Post date:** [November 13, 2021, 1:24am UTC](https://discuss.pytorch.org/t/simple-ann-model-not-training/136434/4 "2021-11-13T01:24:21Z")

</div>

```auto
import numpy as np
import pandas as pd
import os  

from PIL import Image  

import torch  
import torch.nn as nn
import torch.utils.data as data_utils
from torch.optim.lr_scheduler import ExponentialLR
import torch.nn.functional as F 

from torchvision import models, transforms as T

from tqdm import tqdm_notebook

from glob import glob
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split

import matplotlib.pyplot as plt
import seaborn as sns
sns.set(style = "darkgrid")
%matplotlib inline

import warnings
warnings.filterwarnings("ignore")

from ignite.engine import Events, create_supervised_evaluator, create_supervised_trainer
from ignite.metrics import Loss, Accuracy
from ignite.contrib.handlers.tqdm_logger import ProgressBar
from ignite.handlers import EarlyStopping, ModelCheckpoint

target_col = ["target"]
features_col = ["feature1", "feature2", "feature3",...,"feature966"]

def df_to_tensor_w_target_and_features(df):
    target = torch.tensor(df[target_col].values, dtype=torch.float32)
    features = torch.tensor(df[features_col].values, dtype=torch.float32)
    combined = data_utils.TensorDataset(features, target)
    return combined

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
BATCH_SIZE = 64

df = pd.read_csv("./train.dat", sep="\t")
df_test = pd.read_csv("./test.dat", sep="\t")

train_data, val_data = train_test_split(df, random_state = 42, test_size = 0.2, stratify = df["avg_grp_no"])

train = df_to_tensor_w_target_and_features(train_data)
val = df_to_tensor_w_target_and_features(val_data)
test = df_to_tensor_w_target_and_features(df_test)

train_loader = data_utils.DataLoader(train, batch_size=BATCH_SIZE, shuffle=True)
val_loader = data_utils.DataLoader(val, batch_size=BATCH_SIZE, shuffle=False)
test_loader = data_utils.DataLoader(test, batch_size=BATCH_SIZE, shuffle=True)

class Net(nn.Module):
    def __init__ (self):
        super(Net, self). __init__ ()
        
        self.fc1 = nn.Linear(966, 1024)
        self.fc2 = nn.Linear(1024, 256)
        self.fc3 = nn.Linear(256, 10)
        self.fc4 = nn.Linear(10, 1)
        
        self.drpout = nn.Dropout(0.2)

    def forward(self, x):
        x = x.view(-1, 966)
        x = F.relu(self.fc1(x))
        x = self.drpout(x)
        x = F.relu(self.fc2(x))
        x = self.drpout(x)
        x = F.relu(self.fc3(x))
        x = self.fc4(x)
        return x  
    
model = Net()

criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(),lr = 0.001)

metrics = { "loss" : Loss(criterion)}
trainer = create_supervised_trainer(model, optimizer, criterion, device = device)
val_eval = create_supervised_evaluator(model, metrics = metrics, device = device)

@trainer.on(Events.EPOCH_COMPLETED)
def compute_and_display(engine):
    epoch = engine.state.epoch
    metrics = val_eval.run(val_loader).metrics
    print("Validation Results - Epoch: {} Average Loss: {:.4f} ".format(engine.state.epoch, metrics['loss']))
    
handler = EarlyStopping(patience = 40, score_function = lambda engine : engine.state.metrics['loss'], trainer = trainer)
val_eval.add_event_handler(Events.COMPLETED, handler)

checkpoints = ModelCheckpoint("models_1", "Model_1", n_saved = 3, create_dir = True, require_empty=False)
trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpoints, {"ANN" : model})

pbar = ProgressBar(bar_format = '')
pbar.attach(trainer, output_transform = lambda x : {"loss" : x})

trainer.run(train_loader, max_epochs = 4)

```

---

<div class="post-metadata">

**Author:** ![paragG](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/ebca7d/32.png) [@paragG](https://discuss.pytorch.org/u/paragG)\
**Post date:** [November 13, 2021, 1:45am UTC](https://discuss.pytorch.org/t/simple-ann-model-not-training/136434/5 "2021-11-13T01:45:56Z")

</div>

@my3bikaht , @vfdev-5 , I am going to try posting on google colab. I have the code in Kaggle and can share the link if that helps. I apologize for the poor readability. Since my issue is not resolved, I would try to post it in a format that is more readable. Meanwhile, thanks for looking into it.

---

<div class="post-metadata">

**Author:** ![paragG](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/ebca7d/32.png) [@paragG](https://discuss.pytorch.org/u/paragG)\
**Post date:** [November 17, 2021, 7:27pm UTC](https://discuss.pytorch.org/t/simple-ann-model-not-training/136434/6 "2021-11-17T19:27:18Z")

</div>

I found the bug in the code above. I needed to unsqueeze the input target tensors by the adding the line: target = target.unsqueeze(1) This changed the shape of the target tensor from torch.Size([num\_of\_rows]) to torch.Size([num\_of\_rows], **1** ) . The model trains fine now:  
 ![image](https://discuss.pytorch.org/uploads/default/original/3X/d/f/dfe33db663c9c0bc52ab5f8451eb38cc816101c2.png)  
Thanks to those who looked to help.