Precognition: Science or Pseudoscience?
On this page
Can scientists cheat without knowing they're cheating?
Imagine you're a researcher studying whether people can sense things beyond normal perception. You run an experiment, analyze the data, and find... nothing significant. But wait—what if you try a slightly different statistical approach? Suddenly, you have compelling results. This scenario plays out in psychology labs worldwide, and a groundbreaking 2012 study used extrasensory perception research to expose a fundamental flaw in how science validates extraordinary claims. The findings reveal something unsettling about the very foundation of psychological research.
Researchers propose a solution to unconscious bias in data analysis.
In 2012, a team of Dutch psychologists noticed a troubling pattern across their field. Researchers were unconsciously tweaking their data analyses after seeing results, potentially invalidating thousands of studies. They decided to propose a systematic solution using an unlikely test case: extrasensory perception research.
Most psychological research lacks true scientific rigor because researchers choose their analysis methods after seeing the data, making statistical tests meaningless.
Key Findings
- The authors identified a widespread problem: researchers routinely adjust their analyses after seeing data, often unconsciously seeking patterns that support their hypotheses.
- This practice, while not intentionally deceptive, undermines the mathematical foundations of statistical testing.
- Their proposed preregistration system would clearly separate planned 'confirmatory' analyses from unplanned 'exploratory' ones.
What Is This About?
The researchers didn't conduct a traditional experiment. Instead, they analyzed how scientists typically handle data and proposed a new system called 'preregistration.' This means researchers must publicly declare their analysis plan before collecting any data, like announcing your strategy before playing a game. They demonstrated this approach by designing a rigorous replication of a controversial ESP study, showing exactly how the method would work in practice.
The authors propose a methodological framework for preregistering studies and demonstrate it with a confirmatory replication attempt of an extrasensory perception study.
The paper establishes guidelines for distinguishing confirmatory from exploratory analyses and illustrates the approach using ESP research as an example.
How Good Is the Evidence?
This paper has been cited over 1,100 times since 2012, indicating its major influence on research practices. For comparison, most psychology papers receive fewer than 10 citations, making this one of the most impactful methodological papers of the decade.
Supporters argue this approach would dramatically improve scientific reliability by eliminating unconscious bias and making research more transparent. Critics worry that rigid preregistration might stifle discovery and that the system could be gamed by researchers who submit multiple preregistrations. Most scientists now agree the basic principle is sound, though they debate implementation details.
Mainstream: Preregistration is now standard practice that has revolutionized research reliability across psychology and medicine. Moderate: The approach is valuable but needs flexibility to accommodate genuine discoveries during research. Frontier: While useful, preregistration alone cannot solve deeper issues with how science evaluates evidence.
Common misconception: This paper proves ESP doesn't exist. Reality: The paper uses ESP research as an example to demonstrate better research methods, without taking a position on whether ESP is real.
To fully validate this approach, we'd need evidence that preregistered studies produce more reliable and replicable results than non-preregistered ones, and that the method doesn't stifle important discoveries. This paper provides the theoretical framework and practical demonstration, and subsequent research has largely confirmed its benefits across multiple scientific fields.
We propose that researchers preregister their studies and indicate in advance the analyses they intend to conduct. Only these analyses deserve the label 'confirmatory,' and only for these analyses are the common statistical tests valid.
Stance: Mixed
What Does It Mean?
This study used the controversial field of ESP research as a Trojan horse to revolutionize scientific methodology itself. With over 1,100 citations, it became one of the most influential papers in modern psychology—not for proving or disproving psychic abilities, but for exposing how easily any researcher can unconsciously manipulate data to support their desired conclusions.
It's like the difference between calling your shot in pool versus claiming you meant to make whatever ball happened to go in. Both might result in points, but only the first one counts as skill rather than luck.
If these methodological reforms become standard practice, it could dramatically increase the reliability of psychological research and help distinguish genuine phenomena from statistical artifacts. This approach might finally provide a rigorous framework for investigating controversial topics like extrasensory perception, where extraordinary claims require extraordinary evidence. The implications extend beyond psychology to any field where statistical inference guides scientific conclusions.
The timing of when you decide how to analyze your data matters enormously - decisions made after seeing results are much more likely to be biased than those made beforehand.
Understanding Terms
What This Study Claims
Methodology
Fine-tuning analyses to data in order to obtain desired results invalidates the interpretation of common statistical tests
strongOnly preregistered analyses should be labeled 'confirmatory' while other analyses should be labeled 'exploratory'
moderatePreregistering studies and specifying analyses in advance would remedy problems with data-dependent analysis choices
moderatePsychologists almost never commit to a data analysis method before seeing the actual data, which threatens the validity of statistical tests
moderateThis summary is for general information about current research. It does not constitute medical advice. The scientific interpretation of these results is debated among researchers. If personally affected, please consult qualified professionals.