I’m trying to finetune fastRcnn for object detection on a custom dataset. During each epoch after when I use the optimizer.step() function to update parameters, the model is not working in the evaluation part of the code. However, when I comment the optimizer.step() function I can get the output of the model during the evaluation. I appreciate any comments.
model = faster_rcnn.fasterrcnn_resnet50_fpn(pretrained=True, min_size=800)
num_classes = 31
in_features = model.roi_heads.box_predictor.cls_score.in_features
#replace the pre trained head with a new one
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
epochs = 2
model.to(device)
params = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.SGD(params, lr=0.005, momentum=0.9, weight_decay=0.0005)
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=3, gamma=0.1)
for e in range(epochs):
model.train()
loss_history = list()
for frame, label in training_loader:
label = [{k: v.to(device) for k, v in t.items()} for t in label]
frame = frame.to(device)
losses_dict = model(frame,label)
losses = sum(loss for loss in losses_dict.values())
loss_history.append(losses.cpu().detach().numpy().item())
optimizer.zero_grad()
losses.backward()
optimizer.step()
model.eval()
for data, label in validation_loader:
data = data.to(device)
output = model(data)
print(output)
When I don’t use the optimizer.step() the print function is:
[{‘boxes’: tensor([[ 814.4761, 4.1513, 1232.4717, 750.0000],
[ 553.6875, 124.6697, 1333.0000, 573.3456],
[ 633.8101, 279.4264, 1333.0000, 661.9217],
[ 619.6268, 0.0000, 1055.0597, 739.1709],
[ 440.1299, 210.7000, 1203.8578, 750.0000],
[ 421.0741, 663.9869, 1100.3203, 750.0000],
[ 604.5875, 1.8037, 1333.0000, 325.6623],
[ 349.5059, 146.8048, 807.5912, 750.0000],
[ 937.7691, 210.4664, 1333.0000, 750.0000],
[ 745.6071, 387.2783, 1173.9806, 750.0000]]), ‘labels’: tensor([12, 12, 12, 12, 12, 12, 12, 12, 12, 12]), ‘scores’: tensor([0.0570, 0.0562, 0.0555, 0.0540, 0.0523, 0.0520, 0.0512, 0.0502, 0.0501,
0.0501])}]
When I use the optimizer.step() the print function is:
[{‘boxes’: tensor([], size=(0, 4), grad_fn=), ‘labels’: tensor([], dtype=torch.int64), ‘scores’: tensor([], grad_fn=)}]