Using the gradients in a loss function

take look at Assign manual assigned "grad_output"