CNN trained model doesn't appear to be working

I’ve trained a CNN model and I would like to run the trained model against new data. However, it seems that the trained model isn’t predicting the count correctly as it done during the training. I have a feeling that the model is not using the PTH file. Could someone please advise what I am doing wrong, please?

mport argparse
import datetime
import glob
import os
import random
import shutil
import time
from os.path import join

import numpy as np
import pandas as pd
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from torchvision.transforms import ToTensor
from tqdm import tqdm
import torch.optim as optim

from convnet3_eval import Convnet
from dataset2_eval import CellsDataset

parser = argparse.ArgumentParser('Predicting hits from pixels')
parser.add_argument('name',type=str,help='Name of experiment')
parser.add_argument('data_dir',type=str,help='Path to data directory containing images and gt.csv')
parser.add_argument('--weight_decay',type=float,default=0.0,help='Weight decay coefficient (something like 10^-5)')
parser.add_argument('--lr',type=float,default=0.0001,help='Learning rate')
args = parser.parse_args()

metadata = pd.read_csv(join(args.data_dir,'gt.csv'))
metadata.set_index('filename', inplace=True)


dataset = CellsDataset(args.data_dir,transform=ToTensor(),return_filenames=True)
dataset = DataLoader(dataset,num_workers=4,pin_memory=True)
model_path = '/base_model.pth'

model = Convnet()
optimizer = torch.optim.Adam(model.parameters(),lr=args.lr,weight_decay=args.weight_decay)

for images, paths in tqdm(dataset):

    targets = torch.tensor([metadata['count'][os.path.split(path)[-1]] for path in paths]) # B
    targets = targets.float()

    # code to print training data to a csv file
    filename=CellsDataset(args.data_dir,transform=ToTensor(),return_filenames=True)
    output = model(images) # B x 1 x 9 x 9 (analogous to a heatmap)
    preds = output.sum(dim=[1,2,3]) # predicted cell counts (vector of length B)
    print(preds)
    paths_test = np.array([paths])
    names_preds = np.hstack(paths)
    print(names_preds)                
    df=pd.DataFrame({'Image_Name':names_preds, 'Target':targets.detach(), 'Prediction':preds.detach()})
    print(df) 
    # save image name, targets, and predictions
    df.to_csv(r'model.csv', index=False, mode='a')


model.load_state_dict(torch.load(model_path))
model.eval()

I am assuming this is your code for testing the model whose parameters are saved in base_model.pth.
So I am guessing you want to run this:

With the parameters saved in this:

But you load your model parameters at the very end and do nothing with it:

Try putting these two lines:

After this line:

Also. I think you know this, but this would mean you are trying to load trained parameters into Convnet, to do so the parameters saved in /base_model.pth should have also been training on said Convnet

1 Like

This is the exact issue. Many Thanks for your help.

1 Like