TypeError: unsupported operand type(s) for +: 'MSELoss' and 'float'

1

I made a 10 layers nn, with the MSELoss(), it works normally, but I have to add a regularization term, when I add it to the loss function directly, it report an error :
TypeError: add() received an invalid combination of arguments - got (MSELoss), but expected one of:

  • (Tensor other, float alpha)
  • (float other, float alpha)

when I transpose the regularization to numpy, it error too:
TypeError: unsupported operand type(s) for +: ‘MSELoss’ and ‘float’

I know why it happened, but I don`t know how to correct,

My question is that how to write a regularization?
Thank you!
here is my code:

params=model.state_dict()

for k,v in params.items():
print(k)
n1=params[‘layer1.weight’]
n2=params[‘layer2.weight’]
n3=params[‘layer3.weight’]
n4=params[‘layer4.weight’]
n5=params[‘layer5.weight’]
n6=params[‘layer6.weight’]
n7=params[‘layer7.weight’]
n8=params[‘layer8.weight’]
n9=params[‘layer9.weight’]
n=(
1**4*(params[‘layer1.weight’] -params[‘layer1.weight’] )2 +
0.94*(params[‘layer1.weight’] -params[‘layer1.weight’] )2 +
0.84*(params[‘layer1.weight’] -params[‘layer1.weight’] )2 +
0.74*(params[‘layer1.weight’] -params[‘layer1.weight’] )2 +
0.64*(params[‘layer1.weight’] -params[‘layer1.weight’] )2 +
0.54*(params[‘layer1.weight’] -params[‘layer1.weight’] )2 +
0.44*(params[‘layer1.weight’] -params[‘layer1.weight’] )2 +
0.34*(params[‘layer1.weight’] -params[‘layer1.weight’] )**2
)
E=0.003*n

def train(epoch):
model.train()
criterion = torch.nn.MSELoss()+E

2

It looks like you would like to create a static computation graph, which is then executed (theano-style).
In PyTorch you create a dynamic computation graph, so that each operation is executed immediately (with some exceptions like asynchronous execution of GPU operations).
If you would like to add some value to your criterion, you have to calculate the loss first:

criterion = nn.MSELoss()
...
# In your training loop
output = model(data)
loss = criterion(output, target)
loss = loss + E

Also, have a look at this thread for more information regarding custom regularization.

1 Like