Hi again!
This platform helped me lot in making progress in my school project that I’ve been into since last year. My projects is making a semantic segmentation model (61 classes including the background). I am in the step of solving the current problem I am facing which is class imbalance. One answer/suggestion I got here in the forum is to use weighted cross entropy. I have been trying to implement this but I still have the same output (which is my model tends to classify everything as a background). Today I tried another approach in using weighted cross entropy that I read across this platform too ( Passing the weights to CrossEntropyLoss correctly - PyTorch Forums) but I got an error. Can I ask anyone to help me understand what this error means? And how I might solve it? this is my code snippet for my training part:
weights = [1200, 43, 2, 6, 1, 6, 225, 74, 185, 55, 15, 20, 10, 151, 3, 79, 9, 41, 41, 27, 3, 62, 91, 11, 4, 0, 7, 4, 69, 6, 171, 5, 3, 11, 68, 23, 0, 310, 4, 20, 22, 49, 3, 35, 9, 4, 37, 13, 2, 4, 29, 71, 28, 12, 6, 6, 110, 4, 78, 269, 227]
normedWeights = [1-(x/sum(weights)) for x in weights]
print(normedWeights)
class_weights = torch.FloatTensor(normedWeights).to(device)
import torch.optim as optim
optimizer = optim.Adam(model.parameters(), lr = 0.0001)
loss_function = nn.CrossEntropyLoss(weight=class_weights, reduction = 'none')
from openpyxl import *
import time
BATCH_SIZE = 4
EPOCHS = 10
wb=load_workbook("/content/drive/MyDrive/Thesis_Quilang/Epoch_loss.xlsx")
ws=wb["Sheet1"]
def train(model):
model.train()
for epoch in range(EPOCHS):
for i in tqdm(range(0, len(img_train), BATCH_SIZE)):
batch_img_train = img_train[i:i+BATCH_SIZE].view(-1, 3, 224, 224)
batch_mask_train = mask_train[i:i+BATCH_SIZE].view(-1, 1, 224, 224)
model.zero_grad()
outputs = model(batch_img_train)
loss = loss_function(outputs, batch_mask_train.squeeze(1).long())
loss.backward()
optimizer.step() # Does the update
loss_value = round(float(f"{loss}"), 3)
wcell1=ws.cell(epoch+2,1)
wcell1.value=(f"{epoch}")
wcell2=ws.cell(epoch+2,2)
wcell2.value= (f"{loss_value}")
print(f"Epoch: {epoch}, Loss: {loss}")
Model_Name = f"{loss_value}-model"
wcell_header1 = ws.cell(1,1)
wcell_header1.value = 'Epoch'
wcell_header2 = ws.cell(1,2 )
wcell_header2.value = 'Loss Value'
ws.title = (Model_Name)
wb.save("/content/drive/MyDrive/Thesis_Quilang/Epoch_loss.xlsx")
path = F"/content/drive/MyDrive/Thesis_Quilang/{Model_Name}.pt"
torch.save(model.state_dict(), path)
train(model)
And this is the error:
0%| | 0/338 [00:01<?, ?it/s]
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-28-55655fafc090> in <module>()
45
46
---> 47 train(model)
3 frames
/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in _make_grads(outputs, grads)
48 if out.requires_grad:
49 if out.numel() != 1:
---> 50 raise RuntimeError("grad can be implicitly created only for scalar outputs")
51 new_grads.append(torch.ones_like(out, memory_format=torch.preserve_format))
52 else:
RuntimeError: grad can be implicitly created only for scalar outputs