How to apply L1 and l2 for PyTorch library for overfitting problem

1

Hi to all,
I am a beginner and trying to learn PyTorch, I am facing that my training overfitting and I have tried once the L2 by applying it with optimizer SGD weight_decay = 0.4 and the I used Dropout with my model and I still face the accuracy too bad I don’t know why its bad.

Once I found that should apply the weights of each class. can anyone help me for that.?

Note I have 6 classes which are (0, 1, 2, 3, 6, 7) and
class 0: 14626 samples
class 1: 13010 samples
class 2: 15902 samples
class 3: 13337 samples
class 6: 15079 samples
class 7: 16403 samples

how can I apply the weights?

this is my training code

model = Model()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum = 0.9, weight_decay = 0.4)
criterion = torch.nn.CrossEntropyLoss()
num_epochs = 25

model.train()
for epoch in range(num_epochs):
    correct = 0
    for inputs,labels in train_loader:

        inputs = inputs.float()
        labels = labels.long()
        
        # Feed Forward
        outputs = model(inputs)
                
        # Loss Calculation
        loss_train = criterion(outputs, labels)
        L1_lambda = 0.5

        for name, param,*_ in model.parameters():# Applying for L1 Regularization
                    if 'weight' in name:
                        L1_1 = Variable(param, requires_grad=True)
                        L1_2 = torch.norm(L1_1, 6)
                        L1_3 = L1_lambda * L1_2
                        loss_train = loss_train + L1_3
        # Clear the gradient buffer (we don't want to accumulate gradients)
        optimizer.zero_grad()      

        # Backpropagation 
        loss_train.backward() 
        
        # Weight Update: w <-- w - lr * gradient
        optimizer.step()
        
        #Accuracy
        _, predicted = torch.max(outputs, 1)
        labels1 = labels.long()
        correct += (predicted == labels1).float().sum()
        accuracy = (100 * correct / len(train_dataset))
     
    # Print statistics 
    print("Epoch {}/{}, Loss: {:.3f}, Accuracy: {:.3f}".format(epoch+1,num_epochs, loss_train, accuracy))

Thanks in advance.