How to write custom loss layer in FCN

For one of my problem, I want to use my own loss function. Instead of using autograd for
computing the gradients of my loss function and I want to give the gradients manually for the
loss function (last year). How can I give the gradients manually for the loss function.

In other way, once I get the probability map (probability scores) in the last layer of FCN. I want
to compute my custom loss function and its gradients. How can I give the computed gradients
manually in to the last layer.