Replication Crisis: Difference between revisions

(One intermediate revision by one other user not shown)

Line 1:

''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''

The phrase “replication crisis” is used to describe the growing realization that many published scientific results cannot be reproduced when other researchers try to follow the original methods. The crisis has been discussed most often in two broad domains—psychology and biomedicine/clinical medicine—and the evidence so far suggests that psychology has the larger proportion of non-replicable findings, although medicine has the most high-stakes failures and fraud scandals.

'''Short answer'''

~~Replication rates and headline numbers~~

Quantitatively, psychology shows the lowest large-scale replication rate that has actually been measured (about one-third of tested findings replicated) [1]. Medicine (especially biomedicine) certainly suffers from fraud and non-reproducible results, but the available evidence is more fragmentary; there is no single field-wide replication study of comparable scope. Consequently, most commentators accept that the replication crisis is currently ''better documented''—and appears numerically worse—in psychology, while agreeing that certain sub-fields of medicine (e.g., Alzheimer’s research) may harbour equally serious problems.

* In 2015 the Open Science Collaboration re-ran 100 prominent psychology ~~studies and obtained statistically significant results for only 36 % of them, with effect sizes roughly half of those originally reported [1].~~

* A 2023 review of the ~~psychology literature summarized dozens of~~ large-scale replication ~~projects and estimated~~ that ~~roughly 75 %~~ of ~~headline claims fail to replicate—leading its author to declare that “three out of four claims are false”~~ [2].

* Comparable, ~~systematic across-~~the-~~board numbers for clinical medicine are scarcer. What exists tends to focus on particular subfields rather than the whole~~ of ~~medicine~~. ~~Fraud investigations such as~~ the ~~long~~-~~running Alzheimer’s β-amyloid case demonstrate that some influential biomedical papers can rest on manipulated data [3], but they do not show the overall base rate~~ of ~~failure.~~

* The lack of large-scale replication exercises in medicine ~~makes a direct numerical comparison difficult. However, where replications have been attempted~~ (e.g., ~~pre-clinical drug studies~~)~~, success rates have often been well below 50 %, implying that reproducibility~~ problems ~~extend to medicine even if their exact magnitude is still uncertain [5]~~.

~~Patterns and mechanisms~~

'''What the main sources say'''

~~Psychology~~

~~– Heavy reliance on small samples and flexible analytic choices (“p-hacking”) mean that chance findings can make it into print easily [1][4].~~

~~– Replications are comparatively cheap, so large open projects have accumulated precise estimates of failure rates [1].~~

~~Medicine~~

* The Open Science Collaboration’s 2015 project replicated 100 high-profile psychology papers and reproduced only 36 % of the original significant results [1].

~~– Clinical trials are expensive and involve regulatory oversight, so fewer public replication attempts exist; nevertheless,~~ high-~~impact fraud or unreplicable biomarker work (e.g., in Alzheimer research) shows that~~ the ~~problem can have direct consequences for patients [3][5].~~

~~– Medicine’s incentive structure can put enormous financial value on spectacular but fragile early findings, encouraging selective reporting~~ [5].

~~Do the sources agree?~~

* The “Unsafe Science” analysis argues that, once publication bias is taken into account, roughly 75 % of psychology claims are likely false, framing psychology as “ground zero” of the crisis [2].

* ~~Both the 2015 Science consortium and the 2023 Substack essay argue~~ that ~~psychology’s replicability~~ is ~~particularly poor [1][2].~~

* The New York Times opinion piece stresses that biomedical fraud—while not necessarily common—can derail entire therapeutic areas and waste billions [3].

* Vox’s overview of ~~scientific fraud treats~~ psychology ~~and medicine~~ as ~~jointly troubled fields, calling for stronger sanctions for data fabrication across~~ the ~~board~~ [5].

~~Thus, all authors concede that neither field is immune; they differ~~ on ~~whether psychology’s lack of replication (high false~~-~~discovery rate)~~ is ~~worse~~ than ~~medicine’s mixture of smaller-scale non-~~replication ~~plus occasional blockbuster fraud~~.

* Michael Inzlicht’s 2024 essay focuses on stereotype-threat research; he concludes that the effect is “less robust than believed,” but stresses that some psychological constructs do survive replication attempts, suggesting the situation is “serious yet salvageable” [4].

~~Public-discourse timeline (selected milestones)~~

* In medicine, the 2025 New York Times op-ed chronicles alleged data fabrication in Alzheimer’s amyloid-beta studies, contending that years of drug-development resources were misdirected as a result [3].

~~2015 – Science publishes the Open Science Collaboration’s mass replication of psychology studies~~, ~~igniting mainstream discussion of the crisis [1].~~

~~2020-2022 – Internal critiques of celebrated psychology effects such as stereotype threat appear, arguing that many results are smaller or nonexistent upon re-analysis [4].~~

~~2023 – Popular outlets summarise the replication literature; one Substack article frames~~ the ~~issue as 75 % false claims in psychology [2]; Vox calls for criminal penalties for repeat fraudsters across scientific fields [5].~~

2025 ~~– A~~ New York Times op-ed ~~links a decades~~-~~long Alzheimer~~ drug ~~failure largely to fraudulent microscopy images, sparking debate over oversight in biomedical research~~ [3].

~~Conclusion~~

* Vox’s 2024 feature gathers estimates that fraudulent or irreproducible biomedical findings contribute to treatment delays and avoidable deaths, but offers no systematic replication percentage comparable to the psychology figure [5].

~~Based~~ on the ~~best quantitative~~ evidence ~~now available~~, psychology ~~exhibits~~ the ~~lower replication~~ rate. ~~Medicine~~, ~~however~~, ~~remains vulnerable to~~ high-profile ~~frauds that can dominate an entire therapeutic area~~. ~~Because~~ medicine ~~lacks field~~-wide ~~replication audits on~~ the ~~scale already carried out~~ in psychology, the ~~true gap may narrow—or even reverse—once equally systematic studies are completed~~.

'''Are the views consistent?'''

They converge on the claim that both fields have serious reliability issues, but differ in emphasis. Sources [1], [2], and [4] document psychology’s low replication rates. Sources [3] and [5] argue that medicine’s stakes are higher (patient harm, billions in costs) even if the ''measured'' replication failure rate is not yet pinned down. No source provides evidence that medicine as a whole replicates worse than psychology; rather, the claim is that its failures are more consequential.

'''Factors that make psychology look worse'''

* Abundant field-wide audits (e.g., 2015 Science project) produce hard numbers [1].

* Experiments are often small-sample, low-power, and easier to redo quickly, revealing problems faster [2].

* Publication incentives once favoured surprising results; the discipline now publicly tracks corrections and retractions [4].

'''Factors that obscure medicine’s true rate'''

* Clinical trials are costlier and take years, so systematic replications are rare.

* Regulatory oversight (FDA, EMA, etc.) enforces certain standards, potentially boosting replicability, but also concentrates efforts on late-stage trials that may hide earlier basic-science flaws [5].

* High-profile fraud cases (e.g., Alzheimer’s amyloid imaging) attract media coverage without providing denominator data for the field at large [3].

'''Public discourse timeline'''

2011–2014 Psychology begins adopting preregistration and open-data norms following several high-profile failed replications (not detailed in current sources).

2015 Publication of the Open Science Collaboration study quantifies the problem in psychology (36 % replication rate) [1].

2017–2020 Replication efforts expand to economics, social priming, and some biomedical niches; no medicine-wide project yet.

2024 Michael Inzlicht’s pre-print calls for a “reckoning” but also reform optimism in social psychology [4].

Vox article publicises the human toll of biomedical fraud, pushing the crisis narrative beyond psychology [5].

2025 New York Times op-ed links alleged image manipulation in landmark Alzheimer’s studies to stalled drug development, intensifying concern inside medicine [3].

'''Bottom line'''

With systematic audits showing only ~35 % replication success, psychology presently exhibits the ''clearest and worst-documented'' replication crisis. Medicine’s crisis is better described as ''potentially'' just as serious, but still less quantified; notable fraud cases and the high cost of irreproducibility keep the issue in the spotlight. Future large-scale replication projects in clinical and pre-clinical medicine will be needed before a definitive comparison is possible.

== Sources ==

# https://pubmed.ncbi.nlm.nih.gov/26315443/

# [https://pubmed.ncbi.nlm.nih.gov/26315443/ Estimating the Reproducibility of Psychological Science – ''Science''] (2015 peer-reviewed replication study)

# https://unsafescience.substack.com/p/75-of-psychology-claims-are-false

# [https://unsafescience.substack.com/p/75-of-psychology-claims-are-false ~75 % of Psychology Claims Are False – ''Unsafe Science'' (Substack)] (Opinion / Replication-crisis analysis)

# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research - The New York Times]

# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research – ''The New York Times''] (2025 Opinion / Op-Ed)

# https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf

# [https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf Revisiting Stereotype Threat: A Reckoning for Social Psychology – Michael Inzlicht] (2024 pre-print PDF; Scholarly essay)

# https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time

# [https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time The Staggering Death Toll of Scientific Lies – ''Vox''] (2024 explanatory / analysis article)

== Question ==

Is the replication crisis worst in psychology or medicine?

@@ Line 1: / Line 1: @@
 ''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''
-The phrase “replication crisis” is used to describe the growing realization that many published scientific results cannot be reproduced when other researchers try to follow the original methods.  The crisis has been discussed most often in two broad domains—psychology and biomedicine/clinical medicine—and the evidence so far suggests that psychology has the larger proportion of non-replicable findings, although medicine has the most high-stakes failures and fraud scandals.
+'''Short answer'''
-Replication rates and headline numbers
+Quantitatively, psychology shows the lowest large-scale replication rate that has actually been measured (about one-third of tested findings replicated) [1]. Medicine (especially biomedicine) certainly suffers from fraud and non-reproducible results, but the available evidence is more fragmentary; there is no single field-wide replication study of comparable scope. Consequently, most commentators accept that the replication crisis is currently ''better documented''—and appears numerically worse—in psychology, while agreeing that certain sub-fields of medicine (e.g., Alzheimer’s research) may harbour equally serious problems.
-* In 2015 the Open Science Collaboration re-ran 100 prominent psychology studies and obtained statistically significant results for only 36 % of them, with effect sizes roughly half of those originally reported [1].
-* A 2023 review of the psychology literature summarized dozens of large-scale replication projects and estimated that roughly 75 % of headline claims fail to replicate—leading its author to declare that “three out of four claims are false” [2].
-* Comparable, systematic across-the-board numbers for clinical medicine are scarcer.  What exists tends to focus on particular subfields rather than the whole of medicine.  Fraud investigations such as the long-running Alzheimer’s β-amyloid case demonstrate that some influential biomedical papers can rest on manipulated data [3], but they do not show the overall base rate of failure.
-* The lack of large-scale replication exercises in medicine makes a direct numerical comparison difficult.  However, where replications have been attempted (e.g., pre-clinical drug studies), success rates have often been well below 50 %, implying that reproducibility problems extend to medicine even if their exact magnitude is still uncertain [5].
-Patterns and mechanisms
+'''What the main sources say'''
-Psychology
-– Heavy reliance on small samples and flexible analytic choices (“p-hacking”) mean that chance findings can make it into print easily [1][4].
-– Replications are comparatively cheap, so large open projects have accumulated precise estimates of failure rates [1].
-Medicine
+* The Open Science Collaboration’s 2015 project replicated 100 high-profile psychology papers and reproduced only 36 % of the original significant results [1].
-– Clinical trials are expensive and involve regulatory oversight, so fewer public replication attempts exist; nevertheless, high-impact fraud or unreplicable biomarker work (e.g., in Alzheimer research) shows that the problem can have direct consequences for patients [3][5].
-– Medicine’s incentive structure can put enormous financial value on spectacular but fragile early findings, encouraging selective reporting [5].
-Do the sources agree?
+* The “Unsafe Science” analysis argues that, once publication bias is taken into account, roughly 75 % of psychology claims are likely false, framing psychology as “ground zero” of the crisis [2].
-* Both the 2015 Science consortium and the 2023 Substack essay argue that psychology’s replicability is particularly poor [1][2].
-* The New York Times opinion piece stresses that biomedical fraud—while not necessarily common—can derail entire therapeutic areas and waste billions [3].
-* Vox’s overview of scientific fraud treats psychology and medicine as jointly troubled fields, calling for stronger sanctions for data fabrication across the board [5].
-Thus, all authors concede that neither field is immune; they differ on whether psychology’s lack of replication (high false-discovery rate) is worse than medicine’s mixture of smaller-scale non-replication plus occasional blockbuster fraud.
+* Michael Inzlicht’s 2024 essay focuses on stereotype-threat research; he concludes that the effect is “less robust than believed,” but stresses that some psychological constructs do survive replication attempts, suggesting the situation is “serious yet salvageable” [4].
-Public-discourse timeline (selected milestones)
+* In medicine, the 2025 New York Times op-ed chronicles alleged data fabrication in Alzheimer’s amyloid-beta studies, contending that years of drug-development resources were misdirected as a result [3].
-– Science publishes the Open Science Collaboration’s mass replication of psychology studies, igniting mainstream discussion of the crisis [1].
--2022 – Internal critiques of celebrated psychology effects such as stereotype threat appear, arguing that many results are smaller or nonexistent upon re-analysis [4].
-– Popular outlets summarise the replication literature; one Substack article frames the issue as 75 % false claims in psychology [2]; Vox calls for criminal penalties for repeat fraudsters across scientific fields [5].
-– A New York Times op-ed links a decades-long Alzheimer drug failure largely to fraudulent microscopy images, sparking debate over oversight in biomedical research [3].
-Conclusion
+* Vox’s 2024 feature gathers estimates that fraudulent or irreproducible biomedical findings contribute to treatment delays and avoidable deaths, but offers no systematic replication percentage comparable to the psychology figure [5].
-Based on the best quantitative evidence now available, psychology exhibits the lower replication rate.  Medicine, however, remains vulnerable to high-profile frauds that can dominate an entire therapeutic area.  Because medicine lacks field-wide replication audits on the scale already carried out in psychology, the true gap may narrow—or even reverse—once equally systematic studies are completed.
+'''Are the views consistent?'''
+They converge on the claim that both fields have serious reliability issues, but differ in emphasis. Sources [1], [2], and [4] document psychology’s low replication rates. Sources [3] and [5] argue that medicine’s stakes are higher (patient harm, billions in costs) even if the ''measured'' replication failure rate is not yet pinned down. No source provides evidence that medicine as a whole replicates worse than psychology; rather, the claim is that its failures are more consequential.
+'''Factors that make psychology look worse'''
+* Abundant field-wide audits (e.g., 2015 Science project) produce hard numbers [1].
+* Experiments are often small-sample, low-power, and easier to redo quickly, revealing problems faster [2].
+* Publication incentives once favoured surprising results; the discipline now publicly tracks corrections and retractions [4].
+'''Factors that obscure medicine’s true rate'''
+* Clinical trials are costlier and take years, so systematic replications are rare.
+* Regulatory oversight (FDA, EMA, etc.) enforces certain standards, potentially boosting replicability, but also concentrates efforts on late-stage trials that may hide earlier basic-science flaws [5].
+* High-profile fraud cases (e.g., Alzheimer’s amyloid imaging) attract media coverage without providing denominator data for the field at large [3].
+'''Public discourse timeline'''
+–2014 Psychology begins adopting preregistration and open-data norms following several high-profile failed replications (not detailed in current sources).
+ Publication of the Open Science Collaboration study quantifies the problem in psychology (36 % replication rate) [1].
+–2020 Replication efforts expand to economics, social priming, and some biomedical niches; no medicine-wide project yet.
+ Michael Inzlicht’s pre-print calls for a “reckoning” but also reform optimism in social psychology [4].
+   Vox article publicises the human toll of biomedical fraud, pushing the crisis narrative beyond psychology [5].
+ New York Times op-ed links alleged image manipulation in landmark Alzheimer’s studies to stalled drug development, intensifying concern inside medicine [3].
+'''Bottom line'''
+With systematic audits showing only ~35 % replication success, psychology presently exhibits the ''clearest and worst-documented'' replication crisis. Medicine’s crisis is better described as ''potentially'' just as serious, but still less quantified; notable fraud cases and the high cost of irreproducibility keep the issue in the spotlight. Future large-scale replication projects in clinical and pre-clinical medicine will be needed before a definitive comparison is possible.
 == Sources ==
-# https://pubmed.ncbi.nlm.nih.gov/26315443/
+# [https://pubmed.ncbi.nlm.nih.gov/26315443/ Estimating the Reproducibility of Psychological Science – ''Science''] (2015 peer-reviewed replication study)
-# https://unsafescience.substack.com/p/75-of-psychology-claims-are-false
+# [https://unsafescience.substack.com/p/75-of-psychology-claims-are-false ~75 % of Psychology Claims Are False – ''Unsafe Science'' (Substack)] (Opinion / Replication-crisis analysis)
-# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research - The New York Times]
+# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research – ''The New York Times''] (2025 Opinion / Op-Ed)
-# https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf
+# [https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf Revisiting Stereotype Threat: A Reckoning for Social Psychology – Michael Inzlicht] (2024 pre-print PDF; Scholarly essay)
-# https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time
+# [https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time The Staggering Death Toll of Scientific Lies – ''Vox''] (2024 explanatory / analysis article)
 == Question ==
 Is the replication crisis worst in psychology or medicine?