Loss function on the mean of model outputs

autograd
1

Hello everybody. I have a problem where a patient can have multiple images from a study and to determine if the patient has a disease or not we average all the predictions from his individual images. So to summarize:

  1. Iterate over all the patient’s images.
  2. Predict whether the disease is present on each image.
  3. Take the mean of all predictions as the final prediction.
  4. Compute loss function by comparing the mean of predictions with ground truth.
    The code looks like this:
# Get predictions in the form of logits
out = model(images)
# Sigmoid predictions to get probabilities from logits
pred = torch.sigmoid(out)
# Average all predictions for a single patient
pred = torch.mean(pred)
loss = criterion(pred, label)
loss.backward()

I do not know if back propagation is being computed on the mean of the output (called “pred” above) since I have not utilized the requires_grad parameter on torch.mean() nor on torch.sigmoid().

Thanks in advance.

2

Ideally, you do not need to use requires_grad for intermediate result variables.
The model parameters act as leaf variables and typically have requires_grad set to True.

Pytorch automatically takes care of backpropagating the gradients through intermediate results to those leaf variables (i.e., parameters), given that you do not break the computation graph.

1 Like