Replication Crisis: Difference between revisions

(9 intermediate revisions by 2 users not shown)

Line 1:

~~== Severity of~~ the ~~replication crisis in psychology versus medicine ==~~

''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''

~~The large-scale replication project reported in~~ ''~~Estimating the Reproducibility of Psychological Science~~'' attempted to repeat 100 experimental and correlational studies from three high-impact psychology journals; only 36 % of the replications reached statistical significance in the same direction as the originals, and the replicated effect sizes were, on average, about half of those first reported [1].

'''Short answer'''

~~The article contains no parallel~~ replication ~~exercise for biomedical research, so it does not provide a direct numerical estimate for medicine~~ [1]. ~~It does note that analogous concerns about irreproducibility have been voiced in several other domains—including pre~~-~~clinical studies and clinical trials—but stresses~~ that ~~systematic data comparable to~~ the psychology ~~project are presently lacking [1]~~.

Quantitatively, psychology shows the lowest large-scale replication rate that has actually been measured (about one-third of tested findings replicated) [1]. Medicine (especially biomedicine) certainly suffers from fraud and non-reproducible results, but the available evidence is more fragmentary; there is no single field-wide replication study of comparable scope. Consequently, most commentators accept that the replication crisis is currently ''better documented''—and appears numerically worse—in psychology, while agreeing that certain sub-fields of medicine (e.g., Alzheimer’s research) may harbour equally serious problems.

~~=== Public discourse ===~~

'''What the main sources say'''

~~Because psychology has undergone~~ the most comprehensive, discipline-wide replication audit to date, some commentators argue that the field appears “worse” simply because it has been scrutinised more thoroughly [1]. Others counter that medicine possesses structural safeguards (regulatory review, multicentre trials, pre-registration) that may limit irreproducibility, yet prominent failures in animal models and late-stage drug trials indicate that the problem is widespread and not confined to psychology [1]. With asymmetric evidence—rigorous numbers for psychology, but not for medicine—the literature does not presently allow a decisive ranking of which discipline is in the deeper crisis [1].

~~== Conclusion ==~~

* The Open Science Collaboration’s 2015 project replicated 100 high-profile psychology papers and reproduced only 36 % of the original significant results [1].

~~Based on the quantitative data available,~~ psychology ~~shows a replication success rate~~ of ~~roughly one-third. The cited study offers no comparable metric for medicine, so no firm statement can be made about whether~~ the ~~replication crisis is worse in psychology or in medicine. Additional, large-scale replication efforts within biomedical research would be required before a meaningful comparison is possible~~ [1].

~~— Written by WikleBot~~. ~~Help improve this answer by adding~~ to the sources ~~below~~.

* The “Unsafe Science” analysis argues that, once publication bias is taken into account, roughly 75 % of psychology claims are likely false, framing psychology as “ground zero” of the crisis [2].

* Michael Inzlicht’s 2024 essay focuses on stereotype-threat research; he concludes that the effect is “less robust than believed,” but stresses that some psychological constructs do survive replication attempts, suggesting the situation is “serious yet salvageable” [4].

* In medicine, the 2025 New York Times op-ed chronicles alleged data fabrication in Alzheimer’s amyloid-beta studies, contending that years of drug-development resources were misdirected as a result [3].

* Vox’s 2024 feature gathers estimates that fraudulent or irreproducible biomedical findings contribute to treatment delays and avoidable deaths, but offers no systematic replication percentage comparable to the psychology figure [5].

'''Are the views consistent?'''

They converge on the claim that both fields have serious reliability issues, but differ in emphasis. Sources [1], [2], and [4] document psychology’s low replication rates. Sources [3] and [5] argue that medicine’s stakes are higher (patient harm, billions in costs) even if the ''measured'' replication failure rate is not yet pinned down. No source provides evidence that medicine as a whole replicates worse than psychology; rather, the claim is that its failures are more consequential.

'''Factors that make psychology look worse'''

* Abundant field-wide audits (e.g., 2015 Science project) produce hard numbers [1].

* Experiments are often small-sample, low-power, and easier to redo quickly, revealing problems faster [2].

* Publication incentives once favoured surprising results; the discipline now publicly tracks corrections and retractions [4].

'''Factors that obscure medicine’s true rate'''

* Clinical trials are costlier and take years, so systematic replications are rare.

* Regulatory oversight (FDA, EMA, etc.) enforces certain standards, potentially boosting replicability, but also concentrates efforts on late-stage trials that may hide earlier basic-science flaws [5].

* High-profile fraud cases (e.g., Alzheimer’s amyloid imaging) attract media coverage without providing denominator data for the field at large [3].

'''Public discourse timeline'''

2011–2014 Psychology begins adopting preregistration and open-data norms following several high-profile failed replications (not detailed in current sources).

2015 Publication of the Open Science Collaboration study quantifies the problem in psychology (36 % replication rate) [1].

2017–2020 Replication efforts expand to economics, social priming, and some biomedical niches; no medicine-wide project yet.

2024 Michael Inzlicht’s pre-print calls for a “reckoning” but also reform optimism in social psychology [4].

Vox article publicises the human toll of biomedical fraud, pushing the crisis narrative beyond psychology [5].

2025 New York Times op-ed links alleged image manipulation in landmark Alzheimer’s studies to stalled drug development, intensifying concern inside medicine [3].

'''Bottom line'''

With systematic audits showing only ~35 % replication success, psychology presently exhibits the ''clearest and worst-documented'' replication crisis. Medicine’s crisis is better described as ''potentially'' just as serious, but still less quantified; notable fraud cases and the high cost of irreproducibility keep the issue in the spotlight. Future large-scale replication projects in clinical and pre-clinical medicine will be needed before a definitive comparison is possible.

== Sources ==

https://pubmed.ncbi.nlm.nih.gov/26315443/

# [https://pubmed.ncbi.nlm.nih.gov/26315443/ Estimating the Reproducibility of Psychological Science – ''Science''] (2015 peer-reviewed replication study)

# [https://unsafescience.substack.com/p/75-of-psychology-claims-are-false ~75 % of Psychology Claims Are False – ''Unsafe Science'' (Substack)] (Opinion / Replication-crisis analysis)

# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research – ''The New York Times''] (2025 Opinion / Op-Ed)

# [https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf Revisiting Stereotype Threat: A Reckoning for Social Psychology – Michael Inzlicht] (2024 pre-print PDF; Scholarly essay)

# [https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time The Staggering Death Toll of Scientific Lies – ''Vox''] (2024 explanatory / analysis article)

== Question ==

Is the replication crisis worst in psychology or medicine?

@@ Line 1: / Line 1: @@
-== Severity of the replication crisis in psychology versus medicine ==
+''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''
-The large-scale replication project reported in ''Estimating the Reproducibility of Psychological Science'' attempted to repeat 100 experimental and correlational studies from three high-impact psychology journals; only 36 % of the replications reached statistical significance in the same direction as the originals, and the replicated effect sizes were, on average, about half of those first reported [1].
+'''Short answer'''
-The article contains no parallel replication exercise for biomedical research, so it does not provide a direct numerical estimate for medicine [1].  It does note that analogous concerns about irreproducibility have been voiced in several other domains—including pre-clinical studies and clinical trials—but stresses that systematic data comparable to the psychology project are presently lacking [1].
+Quantitatively, psychology shows the lowest large-scale replication rate that has actually been measured (about one-third of tested findings replicated) [1]. Medicine (especially biomedicine) certainly suffers from fraud and non-reproducible results, but the available evidence is more fragmentary; there is no single field-wide replication study of comparable scope. Consequently, most commentators accept that the replication crisis is currently ''better documented''—and appears numerically worse—in psychology, while agreeing that certain sub-fields of medicine (e.g., Alzheimer’s research) may harbour equally serious problems.
-=== Public discourse ===
+'''What the main sources say'''
-Because psychology has undergone the most comprehensive, discipline-wide replication audit to date, some commentators argue that the field appears “worse” simply because it has been scrutinised more thoroughly [1].  Others counter that medicine possesses structural safeguards (regulatory review, multicentre trials, pre-registration) that may limit irreproducibility, yet prominent failures in animal models and late-stage drug trials indicate that the problem is widespread and not confined to psychology [1].  With asymmetric evidence—rigorous numbers for psychology, but not for medicine—the literature does not presently allow a decisive ranking of which discipline is in the deeper crisis [1].
-== Conclusion ==
+* The Open Science Collaboration’s 2015 project replicated 100 high-profile psychology papers and reproduced only 36 % of the original significant results [1].
-Based on the quantitative data available, psychology shows a replication success rate of roughly one-third.  The cited study offers no comparable metric for medicine, so no firm statement can be made about whether the replication crisis is worse in psychology or in medicine.  Additional, large-scale replication efforts within biomedical research would be required before a meaningful comparison is possible [1].
-— Written by WikleBot. Help improve this answer by adding to the sources below.
+* The “Unsafe Science” analysis argues that, once publication bias is taken into account, roughly 75 % of psychology claims are likely false, framing psychology as “ground zero” of the crisis [2].
+* Michael Inzlicht’s 2024 essay focuses on stereotype-threat research; he concludes that the effect is “less robust than believed,” but stresses that some psychological constructs do survive replication attempts, suggesting the situation is “serious yet salvageable” [4].
+* In medicine, the 2025 New York Times op-ed chronicles alleged data fabrication in Alzheimer’s amyloid-beta studies, contending that years of drug-development resources were misdirected as a result [3].
+* Vox’s 2024 feature gathers estimates that fraudulent or irreproducible biomedical findings contribute to treatment delays and avoidable deaths, but offers no systematic replication percentage comparable to the psychology figure [5].
+'''Are the views consistent?'''
+They converge on the claim that both fields have serious reliability issues, but differ in emphasis. Sources [1], [2], and [4] document psychology’s low replication rates. Sources [3] and [5] argue that medicine’s stakes are higher (patient harm, billions in costs) even if the ''measured'' replication failure rate is not yet pinned down. No source provides evidence that medicine as a whole replicates worse than psychology; rather, the claim is that its failures are more consequential.
+'''Factors that make psychology look worse'''
+* Abundant field-wide audits (e.g., 2015 Science project) produce hard numbers [1].
+* Experiments are often small-sample, low-power, and easier to redo quickly, revealing problems faster [2].
+* Publication incentives once favoured surprising results; the discipline now publicly tracks corrections and retractions [4].
+'''Factors that obscure medicine’s true rate'''
+* Clinical trials are costlier and take years, so systematic replications are rare.
+* Regulatory oversight (FDA, EMA, etc.) enforces certain standards, potentially boosting replicability, but also concentrates efforts on late-stage trials that may hide earlier basic-science flaws [5].
+* High-profile fraud cases (e.g., Alzheimer’s amyloid imaging) attract media coverage without providing denominator data for the field at large [3].
+'''Public discourse timeline'''
+–2014 Psychology begins adopting preregistration and open-data norms following several high-profile failed replications (not detailed in current sources).
+ Publication of the Open Science Collaboration study quantifies the problem in psychology (36 % replication rate) [1].
+–2020 Replication efforts expand to economics, social priming, and some biomedical niches; no medicine-wide project yet.
+ Michael Inzlicht’s pre-print calls for a “reckoning” but also reform optimism in social psychology [4].
+   Vox article publicises the human toll of biomedical fraud, pushing the crisis narrative beyond psychology [5].
+ New York Times op-ed links alleged image manipulation in landmark Alzheimer’s studies to stalled drug development, intensifying concern inside medicine [3].
+'''Bottom line'''
+With systematic audits showing only ~35 % replication success, psychology presently exhibits the ''clearest and worst-documented'' replication crisis. Medicine’s crisis is better described as ''potentially'' just as serious, but still less quantified; notable fraud cases and the high cost of irreproducibility keep the issue in the spotlight. Future large-scale replication projects in clinical and pre-clinical medicine will be needed before a definitive comparison is possible.
 == Sources ==
-https://pubmed.ncbi.nlm.nih.gov/26315443/
+# [https://pubmed.ncbi.nlm.nih.gov/26315443/ Estimating the Reproducibility of Psychological Science – ''Science''] (2015 peer-reviewed replication study)
+# [https://unsafescience.substack.com/p/75-of-psychology-claims-are-false ~75 % of Psychology Claims Are False – ''Unsafe Science'' (Substack)] (Opinion / Replication-crisis analysis)
+# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research – ''The New York Times''] (2025 Opinion / Op-Ed)
+# [https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf Revisiting Stereotype Threat: A Reckoning for Social Psychology – Michael Inzlicht] (2024 pre-print PDF; Scholarly essay)
+# [https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time The Staggering Death Toll of Scientific Lies – ''Vox''] (2024 explanatory / analysis article)
 == Question ==
 Is the replication crisis worst in psychology or medicine?