# How to implement an updating weighted MSE Loss?

Hello everyone! I’m new here. I’m an undergraduate student doing my research project.
I’m not a native English speaker, so apologies to my weird grammar.

My research topic is about wind power prediction using an LSTM-NN and its application in the power trading. I used only the time-series date of wind power as parameters, so the power production is predicted based on the observed power production in the past.

Long story short, while trading, we care not only about the forecast accuracy, but also about the power price. When the price is low, a not-very-accurate forecast is acceptable. When the price is high, however, the wind power producer would prefer a highly accurate power production forecast.

In order to achieve this goal, I tried to update my loss function while training. I used the power price as weight factor and multiplied it with my MSE Loss function, so I defined my loss function as below:

``````def weighted_mse_loss(input, target, weight):
return (weight * (input - target) ** 2)
``````

And tried to train my model like this:

``````    # training
loss = 0
for i in range(epochs):
for (seq, label, price_label) in Dtr:
seq = seq.to(device)
label = label.to(device)
y_pred = model(seq)
loss = weighted_mse_loss(y_pred, label, price_label)
loss.backward()
optimizer.step()
print('epoch', i, ':', loss.item())

state = {'model': model.state_dict(), 'optimizer': optimizer.state_dict()}
torch.save(state, path)
``````

The sequence data I input was like this:

``````[(tensor([[-0.4238],
[ 0.7864],
[ 0.7743],
[ 0.6549],
[ 0.7195]]),
tensor([0.7324]),
[6.49]),
``````

At any given time, for example, ([0.7324]) would be the true value of the forecast target, while the previous 5 values would be used for forecast. [6.49] would be the power price at that time which I directly used as the weights.

However, when I run my code, I had an error like this:

``````---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [118], in <cell line: 1>()
----> 1 train(LSTM_PATH)

Input In [117], in train(path)
18 label = label.to(device)
19 y_pred = model(seq)
---> 20 loss = weighted_mse_loss(y_pred, label, price_label)
22 loss.backward()

Input In [109], in weighted_mse_loss(input, target, weight)
1 def weighted_mse_loss(input, target, weight):
----> 2     return (weight * (input - target) ** 2)

TypeError: only integer tensors of a single element can be converted to an index
``````

How could I fix that? Or is there a better way to update my MSE Loss function corresponding to the fluctuation of power price? I’m an EE student so I’m not very good at coding

I’m unsure what exactly is raising the issue as I don’t see an indexing operation in the code.
Also, your code snippet works fine using:

``````def weighted_mse_loss(input, target, weight):
return (weight * (input - target) ** 2)

y = torch.randn(10, 10)
weight = torch.randn(10, 1)

loss = weighted_mse_loss(x, y, weight)
loss.mean().backward()
``````

Could you check the `type` of all inputs to the `weighted_mse_loss` method and post a minimal, executable code snippet here, if possible?

Thank you very much for your response!
The following is the code I used to make the input data:

``````def nn_seq_ms(B):
print('data processing...')
# split
train = dataset[:'2017/12/31 23:50']
test = dataset['2018/1/1 0:00':'2018/12/31 23:50']

def process(data, batch_size):
data = data.values.tolist()
seq = []
for i in range(len(data) - 144): # predict wind power using power production sequence data in the past 144 time intervals
train_seq = [] # sequence data in the past 144 time intervals
train_label = [] # prediction target at any given time
price_label = [] # power price at that same time
for j in range(i, i + 144):
train_seq.append(x)

price_label.append(data[i + 144][0])
train_seq = torch.FloatTensor(train_seq)
train_label = torch.FloatTensor(train_label).view(-1)
#             price_label = torch.FloatTensor(price_label).view(-1)
seq.append((train_seq, train_label, price_label))

# print(seq[-1])
seq = MyDataset(seq)
seq = DataLoader(dataset=seq, batch_size=batch_size, shuffle=False, num_workers=0)

return seq

Dtr = process(train, B)
Dte = process(test, B)

return Dtr, Dte

``````

Ultimately my inputs would be a dataloader, which I’m not sure is the `type` you are asking.

Thanks for the follow-up. Your code doesn’t help much as the `dataset` is undefined and I can’t see its types etc.
Use `print` statements in your code via e.g. `print(type(input), type(target), type(weight))` to check the input types and more attributes if necessary to narrow down what the difference between our code snippet might be.

I’m sorry for confusing you I really suck at coding…
Anyway, the following is my dataset:

The ‘power (kWh)’ is the (normalized) time-series data I used to train the model, while the ‘avg_price (yen/kWh)’ is the power price I want to used as weights.

I’m not sure if that could help you. I’ve also used your code to check my data type and it turned out to be like this:

The error is most likely raised by using a `list` for the `price_label` assuming these are the `weights`:

``````def weighted_mse_loss(input, target, weight):
return (weight * (input - target) ** 2)

y = torch.randn(10, 10)
weight = torch.randn(10, 1).tolist()

loss = weighted_mse_loss(x, y, weight)
# TypeError: only integer tensors of a single element can be converted to an index
``````

Could you transform them to a tensor as it should work based on my previous code snippet?

I’ve transformed the `price_label` to a tensor, but it was still not working. I encountered another error while running the backward:

``````---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [129], in <cell line: 1>()
----> 1 train(LSTM_PATH)

Input In [128], in train(path)
21     loss = weighted_mse_loss(y_pred, label, price_label)
---> 23     loss.backward()
24     optimizer.step()
25 print('epoch', i, ':', loss.item())

File ~\anaconda3\envs\pytorch\lib\site-packages\torch\_tensor.py:396, in Tensor.backward(self, gradient, retain_graph, create_graph, inputs)
387 if has_torch_function_unary(self):
388     return handle_torch_function(
389         Tensor.backward,
390         (self,),
(...)
394         create_graph=create_graph,
395         inputs=inputs)

162 inputs = (inputs,) if isinstance(inputs, torch.Tensor) else \
163     tuple(inputs) if inputs is not None else tuple()
167 if retain_graph is None:
168     retain_graph = create_graph

66     if out.numel() != 1:
---> 67         raise RuntimeError("grad can be implicitly created only for scalar outputs")
69 else:

RuntimeError: grad can be implicitly created only for scalar outputs
``````

Your `loss` tensor contains more than a single element so reduce it e.g. via `loss.mean().backward()`.

Thank you so much! I used `loss.mean().backward()` and it is working!
But I am still a little bit confused about it. While I train the model, the loss is defined by calculating the MSE between the true value and the predicted value, and then multiplied with the weight. I think it will eventually be one single value, so why do I need to reduce it?

The `loss` won’t be automatically reduced and in your `weighted_mse_loss` you are using elementwise operations only.
Check the `loss` output from my first code snippet and you will see that its shape is equal to the input shapes of `[10, 10]` before I call `loss.mean()`.

I fully understand this time! Thank you very much again for being so kind and patient and you really saved my life lol. It seems like I can graduate on time!