I am very new to this pytorch and neural networks.I am stuck in training one model since last 1 week. My model paramters are not getting updated after each epoch. Also, ‘’‘list(model.paramteres()[-1].grad)’’’ returns ‘’‘None’’’.
This is the code I wrote.
for epoch in range(10):
running_loss = 0.0
print(list(net.parameters())[6].grad )
outputs = net(inputs.float())
a=torch.max(outputs,1).indices
a=a.float()
labels=labels.float()
print(list(net.parameters())[6].grad )
loss = criterion(a,labels.squeeze(1))
loss = torch.tensor(loss, requires_grad = True)
print(list(net.parameters())[6].grad )
optimizer.zero_grad()
loss.backward()
optimizer.step()
print(list(net.parameters())[6].grad )
print('Finished Training')
also, this is the optimizer I have used,
import torch.optim as optim
criterion = nn.BCELoss()
optimizer = optim.Adam(net.parameters(), lr=0.001,weight_decay=0.0001)
from torch.autograd import Variable
Basically, what I did is I tried to train the model on only one batch of batch size=16 in order to see if my weights are getting updated or not.And I found that my loss, and the weights all remain same for 100 iterations.I really can’t figure out why is this happening.
I also have another question as well. My dataset contains grayscale images.And I need to perform a binary classification on them. This is the model I have made.It takes as an input tensor of shape (N,1,64,64) where N is the batch_size.
import torch
import torch.nn as nn
import torch.nn.functional as F
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 16, 5)
self.conv2 = nn.Conv2d(16, 32, 7)
self.dropout1 = nn.Dropout2d(0.25)
self.dropout2 = nn.Dropout2d(0.5)
self.fc1 = nn.Linear(4608,128)
self.fc2 = nn.Linear(128,16)
self.fc3 = nn.Linear(16, 2)
def forward(self, x):
# Max pooling over a (2, 2) window
x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
# If the size is a square, you can specify with a single number
x = F.max_pool2d(F.relu(self.conv2(x)), 2)
x = self.dropout1(x)
# x = F.max_pool2d(F.relu(self.conv3(x)), 2)
x = torch.flatten(x, 1) # flatten all dimensions except the batch dimension
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.dropout2(x)
x =self.fc3(x)
#print(x,torch.max(x))
return x
net = Net()
net = net.float()
print(net)