Multiple a real-value quantity

(Aldyarus Chmorius) #1

Hello, I made a model to predict multiple a real-value quantity, exactly two of them, some values could be negative, but now I need to train the model. Which are loss function and optimizers suited for this? Thanks in advance!

(tymokvo) #2

Welcome to the forums!

You have probably noticed that PyTorch can only implicitly create gradients for scalar outputs. Thus, you need to combine them in some way that will make a loss calculation meaningful for your task. A simple objective function like nn.MSELoss might be a good place to start.

You can also take multiple backward passes by specifying output.backward(retain_graph=True)

This is a contrived example that you should be able to translate into your problem:

x = torch.randn(1, 3, requires_grad=True)
y = torch.randn(3, 2, requires_grad=True)

z = x @ y
z0 = z[:, 0]
z1 = z[:, 1]

z0.backward(retain_graph=True)
z1.backward()

Good luck!

2 Likes