My loss function is defined as follows: $Total loss = MSE(w, \phi) + L2norm(F) + MSE(L2norm(\phi), 1). **w** is a d-dimensional vector which I am learning. I want to optimize **w** by taking the gradient of total loss w.r.t **w**. Is there a way to do so without creating a single layer neural network?

You can just define `w`

to be a tensor that requires gradient.

You can instantiate the optimizer with `[w]`

instead of `model.parameters()`

or you could do the optimization yourself in a `torch.no_grad`

block.

Best regards

Thomas