Hey,
for some predictions of my model, I want the optimizer not to consider these predictions in updating the parameters. Therefore, I thought of setting the gradient of the loss corresponding to these predictions to zero.
def loss_fn(y_pred,y):
loss = some_function(y_pred,y)
if y_pred==specific_prediction:
loss.grad = None
return loss
Would that be conceptually correct?
Thanks!
Best, JZ