The same model can receive different inputs
A later query may retrieve an updated record rather than the data available at the original decision time. Re-running the model on that updated record does not reproduce the original situation.
Time-sensitive data access is therefore part of the evidence trail.
Preserve the relevant configuration
- Model and software versions
- Input snapshot or an appropriate traceable reference
- Preprocessing and unit-conversion rules
- Decision threshold and output format
- Retrieval sources or prompts when they affect the task
- Time of execution and relevant error logs
Retention needs to follow applicable privacy, security, and organizational requirements. More logging is not automatically better if it creates unnecessary exposure of sensitive data.
Exact repetition and useful explanation differ
Some systems can vary between runs. Even when an exact output cannot be reproduced, a sufficiently detailed record can support investigation of what information and configuration were used.
Test the investigation process early
Choose a sample output and ask whether the team can trace it end to end. Discovering that a crucial version or input was never retained after an incident is much harder to fix. Reproducibility is an operational capability, not merely a statement that the model has a version number.
