Prediction can rely on a proxy
Suppose a model uses frequent laboratory testing to predict deterioration. Testing frequency may reflect clinicians already being concerned. Reducing tests would not necessarily reduce risk, even if testing appears important to the prediction.
Changing an input in a model is not the same as changing that factor in a patient's life or care.
Ask which explanation is being offered
| Explanation type | What it may describe |
|---|---|
| Feature attribution | Contribution under a particular attribution method |
| Retrieved supporting material | Information supplied alongside the answer |
| Generated narrative | A textual account that needs verification |
None automatically establishes a causal effect.
Evaluate usefulness separately
An explanation may help a user notice a data error or understand the intended use. It may also sound convincing while encouraging overreliance. Whether it improves decisions needs testing with the intended users and task.
For treatment or policy recommendations, causal reasoning requires an appropriate design and assumptions about interventions, confounding, and timing. A colorful importance chart cannot substitute for that work. Keep the claim precise: the model used or emphasized a variable in a particular way; the effect of changing that variable is a separate question.
