Custom Loss function multiple NN outs

I’m trying to train a NN to rank objects. The NN takes as input one object at a time and outputs it’s score. For the loss, I’m trying to implement a function of several outputted scores. However, I’m getting stuck on the loss function because I cant do a backward pass as the resulting loss doesn’t have a gradient because it’s composed of several outputs of the NN. How should I go about implementing this?