Sobolev training loss

#1

I would like to test the loss proposed in the Sobolev training paper. For example consider:

where m(x_i,\theta) is the network evaluated at x_i.

The first part is the usual MSE but I have no idea how to go for the second piece.

(Juan F Montesinos) #2

In practical examples they use first order derivative which actually is what you get with the framework right now. If you are trying to learn a function f (which in this case is a already trained network) you can compute gradients of the output wrt input. and m(x|O) is the network which is learning to approximate f.

In short, they apply an MSE not only over output but over gradients too.

#3

This is my example that employs the derivative error as well as the MSE.

#4

Thank you. This is what I was looking for.

At line 95 in potfit.py you have:

output0.sum().backward(retain_graph=True, create_graph=True)

why the .sum() is needed?

#5

https://pytorch.org/docs/stable/autograd.html#torch.Tensor

If the sum() is not used, the backward() requires a gradient of the scalar function wrt Self.