Pytorch model not converging

vision
1

Hi, I am new to pytorch and deep learning.

The loss of my model was almost static throughout the training p

Model class

class Model(nn.Module):
	def __init__(self):
		super().__init__()
		self.conv1 = nn.Conv2d(3,32,3)
		self.conv2 = nn.Conv2d(32,64,3)
		self.conv3 = nn.Conv2d(64,128,3)
		self.fc1 = nn.Linear(128*2*2,100)
		self.fc2 = nn.Linear(100,10)
	def forward(self,x):
		x = self.conv1(x)
		x = F.relu(x)
		x = F.max_pool2d(x,(2,2))#32x15x15
		
		x = self.conv2(x)
		x = F.relu(x)
		x = F.max_pool2d(x,(2,2))#64x6x6

		x = self.conv3(x)
		x = F.relu(x)
		x = F.max_pool2d(x,(2,2))#128x2x2
		x = x.view(-1,128*2*2)
		x = self.fc1(x)
		x = F.relu(x)
		x = self.fc2(x)
		x = F.softmax(x,dim=1)
		return x

I was training the model on the cifar10 dataset.

train.py

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
import torchvision.transforms as transforms
from cifarloader import cifar10
from model import Model


t = transforms.Compose(
		[
			lambda x:torch.Tensor(x),
			lambda x:x/255,
			transforms.Normalize((0.475,0.475,0.475),(0.253,0.253,0.253))

		]
)


cifar10 = cifar10(transforms=t)
dataloader = DataLoader(cifar10,batch_size=20,shuffle=True)
net = Model()


loss_fn = nn.MSELoss()
optim = optim.Adam(net.parameters(),lr=0.003)


epochs = 100

for e in range(epochs):
	tloss = 0
	for b in dataloader:

		x,y = b
		optim.zero_grad()
		p = net(x)
		loss = loss_fn(p,y)
		tloss+=loss*20
		loss.backward()
		optim.step()
	if e%10 == 0:
		print(f'Epoch: {e}, Loss{tloss/20}')
		torch.save(net,f'e-{e}-l-{tloss}.pth')

Was there something i did wrong, It would be cool if someone could help me out

2

I think I saw the cause of the problem.
However, I had this problem with your code: No module named cifarloader.

Is it a python module (if so how do I install it) or one of your scripts (if so, I need it)?

3

It’s one of my scripts

here’s the code

import torch
from torch.utils.data import Dataset
from data import data,labels
class cifar10(Dataset):
	def __init__(self,transforms):
		self.x = data
		self.y = labels
		self.transforms = transforms
	def __getitem__(self,idx):
		x = self.x[idx].transpose(1,2,0)
		y = self.y[idx]
		if self.transforms:
			x = self.transforms(x)
		return x,y
	def __len__(self):
		return len(self.x)
4

The line from data import data,labels, data is which module?

Help me to help you, by giving me a complete code that I can execute without any problem.

5

Mean Squared Error is not a good loss function for classification (Why MSE Cost Function for Binary Classification is a Bad Idea?)

Replace loss_fn = nn.MSELoss() by loss_fn = nn.CrossEntropyLoss() and let me know if it solved your problem.

6

thanks a lot. I changed the learning rate to 0.0001 and the loss function to cross Entropy. And yeah mse is best suited for regression problems

7

Your problem has been solved?