Two decisions in a fictional study
Suppose a university invites volunteers for a study of two learning formats. After enrollment, a random process assigns each volunteer to one format. Assignment is randomized, but the sample consists of volunteers rather than a random selection of all students.
The comparison may support a causal estimate under the trial's conditions while leaving questions about whether the result applies to students who would not volunteer.
Now imagine randomly sampling students and asking which format they already use. The sample may better represent the population, but students selected their own format. The observed difference can reflect factors other than the format itself.
The words should identify the step
| Phrase | Step it describes |
|---|---|
| Random sample | Selection into the study |
| Random allocation | Assignment within the study |
| Random order | Sequence of tasks or measurements |
“Participants were randomly chosen” is too ambiguous if the paper means that volunteers were assigned to groups. The recruitment and allocation sections resolve the distinction.
Neither process guarantees everything
Random assignment does not guarantee exact baseline balance in a particular sample, prevent all missing data, or remove bias from outcome measurement. Random sampling can still be undermined by nonresponse or incomplete coverage of the target population.
The CDC's field-study guidance and Cochrane's trial-bias guidance address different parts of these design questions. Their separation is useful: representativeness and the credibility of an intervention comparison require related but distinct reasoning.
A concise reading check
Ask who could enter, who did enter, and how the groups were formed. Those three questions often reveal more than the word “random” in a title. They also prevent a strong feature in one part of a design from being treated as a guarantee about the whole study.
