Three questions need separate answers
| Question | What needs evaluation |
|---|---|
| Can people be identified? | Privacy risks and generation process |
| Does the dataset resemble reality? | Relevant distributions and relationships |
| Can it answer this research question? | Preservation of the specific analytic signal |
A dataset can look realistic while failing to preserve an important association. It can also reproduce unusual details too closely if the generation method memorizes source material.
A useful test dataset has a narrower claim
Synthetic records may be valuable for demonstrating a user interface or testing whether software accepts valid formats. Success at that task does not prove that model performance on the synthetic data predicts performance in a real clinic.
Label the intended use clearly.
Ask how the result was checked
The generation method, source data, privacy evaluation, and utility tests all matter. A vendor's use of the word “synthetic” is not a complete explanation.
For an analysis, inspect the relationships that drive the conclusion, including rare groups and missingness patterns. For a privacy decision, use an appropriate assessment rather than visual inspection of a few rows. A convincing-looking patient table can conceal both scientific limitations and disclosure risks that are not visible in the example records.
