IG & KernelShap do not add up to prediction?


Thanks so much for putting together such a great tool. :slight_smile: So I am trying to use integrated gradients and SHAP on a complicated model, and my feature importance scores do not seem to add up to the model output. However, they can be rescaled to add up to the model output.

The model is a causal inference model with multiple inputs and outputs. At the very end, I create a wrapper to output a single quantity, the treatment effect. I’ve got complete code in this notebook.

Do you have any idea why this might be happening? Am I picking the wrong baseline?

Thanks for your help; I really appreciate it!