Loss Variable grad_fn

Hello together,

i try to combine a c++ programm with my neuronal network for training. Therefore i try to set a loss manually, but when i do this and print it out i get different results.

diff = part1 + part2
loss = diff.mean()
tensor(0.2783, device='cuda:0', grad_fn=<MeanBackward0>)

a = torch.Tensor([123]).to("cuda") - torch.Tensor([1]).to("cuda")
tensor([122.], device='cuda:0')

I know that the programm runs through without an error but i dont know how important this grad_fn is.
Can somebody tell me more about this?

grad_fn = <MeanBackward0>

Best regards

The grad_fn is used during the backward() operation for the gradient calculation.
In the first example, at least one of the input tensors (part1 or part2 or both) are attached to a computation graph. Since the loss tensor is calculated from a mean() operation, the grad_fn will point to MeanBackward.
Your second example uses two plain tensors, which do not require gradients, so no computation graph will be created, and thus the grad_fn will be set to None.

thanks for the quick reply. I have looked at the computational graph again.
I have the following problem and I am developing a program in C++ and I want to develop a self-supervised net. However, since I have already pre-trained the net in Python, I wrote an interface using Embedded Python. I am now wondering if I can use C++ to calculate the loss and then convert it to Python and then convert it to a torch variable and then use it to call my backward function. Would this have a negative impact on my training?

I haven’t tried to call Python from C++ and don’t know if this approach would work.
The easier approach might be to script the Python model, load it in your libtorch C++ application and use plain C++ for further processing without going back to Python.


yes i thought so too, however i can convert the neural net to c++ with torchscript but not further train it. If I rebuild the nn in c++, then the loading of the weights should not work.
Do you know more about this?
I would otherwise calculate everything necessary in c++ (without torch) and then convert to python and then to pytorch.