Reading the evidence
Data snooping is repeated exploration of the same dataset until a favourable pattern is found, often without carrying the full search into the reported uncertainty.
Make it click
Testing many keys on one worn lock will eventually produce a movement that looks meaningful. Presenting only that key hides how many chances luck received.
Now put that into markets
Researchers can vary dates, cohorts, indicators and outcomes. The selected rule inherits evidence debt from every prior look at the archive.
Snooping overlaps with multiple testing and overfitting. A holdout loses independence once it guides revisions.
Preserve notebooks, failed specifications and decision logs. Distinguish exploratory patterns from frozen confirmatory tests.
Exploration is necessary for discovery; the error is calling the discovered pattern clean confirmation. Fresh evidence cannot be reconstructed by relabelling old data.
Takeaway
