Psychology's Crisis: Is Observation the Answer?
On this page
Is psychology's favorite statistical method misleading us?
Imagine a field of science suddenly questioning its own methods after a controversial study about predicting the future made headlines. In 2011, psychologist Daryl Bem published research suggesting people could sense future events before they happened, and it passed all the usual scientific checks. The psychology community erupted in debate — not just about the findings, but about whether their entire approach to research was fundamentally flawed. This crisis moment sparked researchers like James Grice to propose a completely different way of doing science.
Researchers propose analyzing data by direct observation rather than abstract statistics.
In 2011, a controversial study claimed evidence for precognition, sparking a crisis in psychology. These authors argue the problem isn't just one study—it's how psychologists analyze all their data, dating back decades.
Sometimes the most important scientific discoveries aren't about what we find, but about realizing our methods for finding things might be broken.
Key Findings
- Traditional statistical methods often obscure cause-and-effect relationships.
- The new OOM method makes causal connections visible and understandable through direct observation, avoiding pitfalls that contributed to psychology's replication crisis.
What Is This About?
The authors developed a new framework called Observation Oriented Modeling (OOM). Instead of relying on complex statistical formulas that compare groups to hypothetical averages, they propose looking directly at patterns in actual data. They tested this approach on three different datasets to show how it reveals causal relationships more clearly than traditional methods.
Theoretical analysis proposing Observation Oriented Modeling as an alternative to traditional statistical methods, with three example demonstrations
OOM demonstrates advantages over null hypothesis significance testing through example applications
How Good Is the Evidence?
Supporters say OOM brings psychology back to observable reality and prevents statistical trickery that can produce false positives. Skeptics worry that abandoning traditional statistics might introduce subjective bias, since 'common sense' varies between observers and lacks the standardization needed for scientific rigor.
Mainstream: Traditional null hypothesis testing remains essential for psychological science. Moderate: OOM offers useful complementary tools for specific research questions where causality is key. Frontier: Psychology must abandon statistical significance testing entirely for observation-based causal modeling.
Many people think statistics prove facts automatically. Actually, this paper shows that different statistical approaches can lead to very different conclusions from the same data, and that 'significant' results don't always mean real effects exist.
To establish OOM as superior to traditional statistics, researchers would need large-scale comparisons showing OOM produces more replicable results across diverse psychological phenomena. This paper provides initial conceptual demonstrations but does not meet these criteria.
Observation Oriented Modeling (OOM) is presented as an alternative approach toward data conceptualization and analysis for the social and life sciences.
Stance: Mixed
What Does It Mean?
The most fascinating aspect is how a study about predicting the future ended up triggering a complete rethinking of how science itself should work. It's like discovering that the telescope you've been using to study the stars has been showing you distorted images all along.
It's like the difference between judging a cake by comparing it to a theoretical 'average cake' versus actually tasting it and observing its specific ingredients and texture.
If Grice's critique is valid, it could mean that decades of psychological research might need fundamental reinterpretation through new analytical lenses. This could potentially rehabilitate controversial findings that were dismissed due to methodological concerns, while also revealing that some accepted findings might be artifacts of flawed analysis. The implications extend beyond psychology to any field relying heavily on statistical significance testing.
The way you analyze data shapes what you can discover—choosing the right statistical framework is as important as collecting the data itself.
Understanding Terms
What This Study Claims
Findings
Three example studies demonstrate OOM's advantages over current researcher practices
weakMethodology
OOM emphasizes causality and common sense over abstract statistical comparison to hypothetical averages
moderateObservation Oriented Modeling provides an alternative approach to data conceptualization rooted in philosophical realism
moderateInterpretations
Serious criticisms of psychology's research practices and data analysis methods date back to at least the mid-1900s
moderateThis summary is for general information about current research. It does not constitute medical advice. The scientific interpretation of these results is debated among researchers. If personally affected, please consult qualified professionals.