A hypothetical deployment report
Suppose 1,000 encounters are eligible. The model receives usable inputs for 850 and produces a result for 800. An accuracy figure calculated on those 800 describes only that subset.
It does not show how the whole intended service performed for all 1,000 encounters.
Keep the stages visible
| Stage | Question |
|---|---|
| Eligible encounters | Who was intended to receive support? |
| Available inputs | Who could be processed? |
| Successful outputs | Who received a result? |
| Delivered outputs | Did the result reach the workflow? |
| Reviewed outputs | Was it actually considered? |
Each transition can fail for different reasons and affect different groups.
Missingness may be selective
If incomplete inputs are more common in one care setting or population, an output-only evaluation can conceal uneven coverage. Report both performance and availability where possible.
Do not confuse coverage with correctness
A system can produce an answer for everyone and be inaccurate. It can also perform well on a narrow subset while serving too few of the intended patients. Both dimensions matter.
Define the denominator before launch, preserve failure reasons, and show changes over time. The resulting report may look less tidy, but it will describe the service people actually receive more faithfully.
