Replication Crisis: Difference between revisions

(3 intermediate revisions by 2 users not shown)

Line 1:

''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''

The question of whether the replication crisis is worse in psychology or in medicine hinges on how “worse” is defined—percent of studies that fail, prominence of the failures, or downstream social cost. The four sources listed focus more heavily on psychology, but they also give a glimpse into biomedical research (specifically Alzheimer’s).

'''Short answer'''

~~Psychology~~

Quantitatively, psychology shows the lowest large-scale replication rate that has actually been measured (about one-third of tested findings replicated) [1]. Medicine (especially biomedicine) certainly suffers from fraud and non-reproducible results, but the available evidence is more fragmentary; there is no single field-wide replication study of comparable scope. Consequently, most commentators accept that the replication crisis is currently ''better documented''—and appears numerically worse—in psychology, while agreeing that certain sub-fields of medicine (e.g., Alzheimer’s research) may harbour equally serious problems.

* In 2015 the ~~Open Science Collaboration attempted to replicate 100 high~~-~~profile psychology papers and could reproduce the original statistical result only 36 %~~ of ~~the time~~ [1].

* A 2023 commentary argues that roughly 75 % of day-to-~~day psychology findings are likely false~~, ~~citing continual failed replications and persistent questionable research practices [2].~~

* A detailed reassessment of stereotype-threat research finds the ~~mean effect dramatically smaller than early reports and documents multiple non-replications~~, ~~highlighting~~ that ~~even celebrated social~~-~~psychological effects are shaky [4]~~.

~~Medicine (biomedical research)~~

'''What the main sources say'''

* A 2025 New York Times opinion piece describes large-scale data fabrication in a landmark line of Alzheimer’s studies; subsequent attempts to reproduce the ~~work have failed and drug programs built on the disputed findings were halted [3].~~

* The article stresses that fraud and irreproducibility in basic biomedical science can derail billion-dollar drug programs and patient hopes, but it does not quantify an overall failure rate for medicine [3].

~~Comparative assessment~~

* The Open Science Collaboration’s 2015 project replicated 100 high-profile psychology papers and reproduced only 36 % of the original significant results [1].

* ~~Quantified evidence: Psychology offers a systematic~~ 100-~~paper audit (~~36 % ~~success) plus meta-analyses of specific paradigms. The biomedical source supplies anecdotes~~ of ~~spectacular failure but no field-wide replication audit.~~

* Severity by proportion of failed studies: On the ~~available numbers, psychology looks worse (≥ 64 % non-replication in a random sample~~ [1] ~~and possibly higher according to the 75 % claim [2]).~~

* Severity by real-world impact: Biomedical failures like the Alzheimer’s case can divert years of drug development and billions of dollars [3], so the societal cost per failed result may be higher even if overall percentages are unknown.

~~Where~~ the ~~authors disagree~~

* The “Unsafe Science” analysis argues that, once publication bias is taken into account, roughly 75 % of psychology claims are likely false, framing psychology as “ground zero” of the crisis [2].

* Sources [1] and [2] ~~both characterise psychology’s problem as widespread and quantifiable.~~

* Source [3] does not claim medicine’s crisis is smaller—it highlights egregious instances and suggests systemic vulnerability—but offers no numerical baseline.

* Because the biomedical piece focuses on fraud whereas the psychology pieces emphasise routine non-replication, they frame “crisis” differently.

~~Public~~-~~discourse timeline~~

* Michael Inzlicht’s 2024 essay focuses on stereotype-threat research; he concludes that the effect is “less robust than believed,” but stresses that some psychological constructs do survive replication attempts, suggesting the situation is “serious yet salvageable” [4].

~~2015 Publication of “Estimating~~ the ~~Reproducibility of Psychological Science~~,” ~~launching mainstream discussion of psychology’s~~ replication ~~crisis [1].~~

~~2018–2022 Large numbers of stereotype-threat and other social-psychology studies fail to replicate; Inzlicht’s overview crystallises doubts [4].~~

~~2023 The Unsafescience newsletter claims three-quarters of everyday psychology claims are false~~, ~~keeping~~ the ~~topic in public view~~ [~~2].~~

~~2025 The New York Times reports on alleged fraud in foundational Alzheimer’s papers, shifting part of the replication conversation toward medicine [3~~].

~~Conclusion~~

* In medicine, the 2025 New York Times op-ed chronicles alleged data fabrication in Alzheimer’s amyloid-beta studies, contending that years of drug-development resources were misdirected as a result [3].

~~With~~ the ~~present~~ evidence, psychology ~~shows~~ the ~~higher documented non~~-replication rate, ~~while~~ medicine ~~demonstrates fewer audited numbers~~ but ~~potentially higher stakes per failure~~. ~~Ongoing~~, ~~field~~-~~wide~~ replication ~~projects akin to psychology’s 2015 audit would be required before one can definitively say~~ the crisis is ~~“worse”~~ in ~~one domain over~~ the ~~other~~.

* Vox’s 2024 feature gathers estimates that fraudulent or irreproducible biomedical findings contribute to treatment delays and avoidable deaths, but offers no systematic replication percentage comparable to the psychology figure [5].

'''Are the views consistent?'''

They converge on the claim that both fields have serious reliability issues, but differ in emphasis. Sources [1], [2], and [4] document psychology’s low replication rates. Sources [3] and [5] argue that medicine’s stakes are higher (patient harm, billions in costs) even if the ''measured'' replication failure rate is not yet pinned down. No source provides evidence that medicine as a whole replicates worse than psychology; rather, the claim is that its failures are more consequential.

'''Factors that make psychology look worse'''

* Abundant field-wide audits (e.g., 2015 Science project) produce hard numbers [1].

* Experiments are often small-sample, low-power, and easier to redo quickly, revealing problems faster [2].

* Publication incentives once favoured surprising results; the discipline now publicly tracks corrections and retractions [4].

'''Factors that obscure medicine’s true rate'''

* Clinical trials are costlier and take years, so systematic replications are rare.

* Regulatory oversight (FDA, EMA, etc.) enforces certain standards, potentially boosting replicability, but also concentrates efforts on late-stage trials that may hide earlier basic-science flaws [5].

* High-profile fraud cases (e.g., Alzheimer’s amyloid imaging) attract media coverage without providing denominator data for the field at large [3].

'''Public discourse timeline'''

2011–2014 Psychology begins adopting preregistration and open-data norms following several high-profile failed replications (not detailed in current sources).

2015 Publication of the Open Science Collaboration study quantifies the problem in psychology (36 % replication rate) [1].

2017–2020 Replication efforts expand to economics, social priming, and some biomedical niches; no medicine-wide project yet.

2024 Michael Inzlicht’s pre-print calls for a “reckoning” but also reform optimism in social psychology [4].

Vox article publicises the human toll of biomedical fraud, pushing the crisis narrative beyond psychology [5].

2025 New York Times op-ed links alleged image manipulation in landmark Alzheimer’s studies to stalled drug development, intensifying concern inside medicine [3].

'''Bottom line'''

With systematic audits showing only ~35 % replication success, psychology presently exhibits the ''clearest and worst-documented'' replication crisis. Medicine’s crisis is better described as ''potentially'' just as serious, but still less quantified; notable fraud cases and the high cost of irreproducibility keep the issue in the spotlight. Future large-scale replication projects in clinical and pre-clinical medicine will be needed before a definitive comparison is possible.

== Sources ==

# https://pubmed.ncbi.nlm.nih.gov/26315443/

# [https://pubmed.ncbi.nlm.nih.gov/26315443/ Estimating the Reproducibility of Psychological Science – ''Science''] (2015 peer-reviewed replication study)

# https://unsafescience.substack.com/p/75-of-psychology-claims-are-false

# [https://unsafescience.substack.com/p/75-of-psychology-claims-are-false ~75 % of Psychology Claims Are False – ''Unsafe Science'' (Substack)] (Opinion / Replication-crisis analysis)

# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research - The New York Times]

# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research – ''The New York Times''] (2025 Opinion / Op-Ed)

# https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf

# [https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf Revisiting Stereotype Threat: A Reckoning for Social Psychology – Michael Inzlicht] (2024 pre-print PDF; Scholarly essay)

# [https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time The Staggering Death Toll of Scientific Lies – ''Vox''] (2024 explanatory / analysis article)

== Question ==

Is the replication crisis worst in psychology or medicine?

@@ Line 1: / Line 1: @@
 ''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''
-The question of whether the replication crisis is worse in psychology or in medicine hinges on how “worse” is defined—percent of studies that fail, prominence of the failures, or downstream social cost. The four sources listed focus more heavily on psychology, but they also give a glimpse into biomedical research (specifically Alzheimer’s).
+'''Short answer'''
-Psychology
+Quantitatively, psychology shows the lowest large-scale replication rate that has actually been measured (about one-third of tested findings replicated) [1]. Medicine (especially biomedicine) certainly suffers from fraud and non-reproducible results, but the available evidence is more fragmentary; there is no single field-wide replication study of comparable scope. Consequently, most commentators accept that the replication crisis is currently ''better documented''—and appears numerically worse—in psychology, while agreeing that certain sub-fields of medicine (e.g., Alzheimer’s research) may harbour equally serious problems.
-* In 2015 the Open Science Collaboration attempted to replicate 100 high-profile psychology papers and could reproduce the original statistical result only 36 % of the time [1].
-* A 2023 commentary argues that roughly 75 % of day-to-day psychology findings are likely false, citing continual failed replications and persistent questionable research practices [2].
-* A detailed reassessment of stereotype-threat research finds the mean effect dramatically smaller than early reports and documents multiple non-replications, highlighting that even celebrated social-psychological effects are shaky [4].
-Medicine (biomedical research)
+'''What the main sources say'''
-* A 2025 New York Times opinion piece describes large-scale data fabrication in a landmark line of Alzheimer’s studies; subsequent attempts to reproduce the work have failed and drug programs built on the disputed findings were halted [3].
-* The article stresses that fraud and irreproducibility in basic biomedical science can derail billion-dollar drug programs and patient hopes, but it does not quantify an overall failure rate for medicine [3].
-Comparative assessment
+* The Open Science Collaboration’s 2015 project replicated 100 high-profile psychology papers and reproduced only 36 % of the original significant results [1].
-* Quantified evidence: Psychology offers a systematic 100-paper audit (36 % success) plus meta-analyses of specific paradigms. The biomedical source supplies anecdotes of spectacular failure but no field-wide replication audit.
-* Severity by proportion of failed studies: On the available numbers, psychology looks worse (≥ 64 % non-replication in a random sample [1] and possibly higher according to the 75 % claim [2]).
-* Severity by real-world impact: Biomedical failures like the Alzheimer’s case can divert years of drug development and billions of dollars [3], so the societal cost per failed result may be higher even if overall percentages are unknown.
-Where the authors disagree
+* The “Unsafe Science” analysis argues that, once publication bias is taken into account, roughly 75 % of psychology claims are likely false, framing psychology as “ground zero” of the crisis [2].
-* Sources [1] and [2] both characterise psychology’s problem as widespread and quantifiable.
-* Source [3] does not claim medicine’s crisis is smaller—it highlights egregious instances and suggests systemic vulnerability—but offers no numerical baseline.
-* Because the biomedical piece focuses on fraud whereas the psychology pieces emphasise routine non-replication, they frame “crisis” differently.
-Public-discourse timeline
+* Michael Inzlicht’s 2024 essay focuses on stereotype-threat research; he concludes that the effect is “less robust than believed,” but stresses that some psychological constructs do survive replication attempts, suggesting the situation is “serious yet salvageable” [4].
- Publication of “Estimating the Reproducibility of Psychological Science,” launching mainstream discussion of psychology’s replication crisis [1].
-–2022 Large numbers of stereotype-threat and other social-psychology studies fail to replicate; Inzlicht’s overview crystallises doubts [4].
- The Unsafescience newsletter claims three-quarters of everyday psychology claims are false, keeping the topic in public view [2].
- The New York Times reports on alleged fraud in foundational Alzheimer’s papers, shifting part of the replication conversation toward medicine [3].
-Conclusion
+* In medicine, the 2025 New York Times op-ed chronicles alleged data fabrication in Alzheimer’s amyloid-beta studies, contending that years of drug-development resources were misdirected as a result [3].
-With the present evidence, psychology shows the higher documented non-replication rate, while medicine demonstrates fewer audited numbers but potentially higher stakes per failure. Ongoing, field-wide replication projects akin to psychology’s 2015 audit would be required before one can definitively say the crisis is “worse” in one domain over the other.
+* Vox’s 2024 feature gathers estimates that fraudulent or irreproducible biomedical findings contribute to treatment delays and avoidable deaths, but offers no systematic replication percentage comparable to the psychology figure [5].
+'''Are the views consistent?'''
+They converge on the claim that both fields have serious reliability issues, but differ in emphasis. Sources [1], [2], and [4] document psychology’s low replication rates. Sources [3] and [5] argue that medicine’s stakes are higher (patient harm, billions in costs) even if the ''measured'' replication failure rate is not yet pinned down. No source provides evidence that medicine as a whole replicates worse than psychology; rather, the claim is that its failures are more consequential.
+'''Factors that make psychology look worse'''
+* Abundant field-wide audits (e.g., 2015 Science project) produce hard numbers [1].
+* Experiments are often small-sample, low-power, and easier to redo quickly, revealing problems faster [2].
+* Publication incentives once favoured surprising results; the discipline now publicly tracks corrections and retractions [4].
+'''Factors that obscure medicine’s true rate'''
+* Clinical trials are costlier and take years, so systematic replications are rare.
+* Regulatory oversight (FDA, EMA, etc.) enforces certain standards, potentially boosting replicability, but also concentrates efforts on late-stage trials that may hide earlier basic-science flaws [5].
+* High-profile fraud cases (e.g., Alzheimer’s amyloid imaging) attract media coverage without providing denominator data for the field at large [3].
+'''Public discourse timeline'''
+–2014 Psychology begins adopting preregistration and open-data norms following several high-profile failed replications (not detailed in current sources).
+ Publication of the Open Science Collaboration study quantifies the problem in psychology (36 % replication rate) [1].
+–2020 Replication efforts expand to economics, social priming, and some biomedical niches; no medicine-wide project yet.
+ Michael Inzlicht’s pre-print calls for a “reckoning” but also reform optimism in social psychology [4].
+   Vox article publicises the human toll of biomedical fraud, pushing the crisis narrative beyond psychology [5].
+ New York Times op-ed links alleged image manipulation in landmark Alzheimer’s studies to stalled drug development, intensifying concern inside medicine [3].
+'''Bottom line'''
+With systematic audits showing only ~35 % replication success, psychology presently exhibits the ''clearest and worst-documented'' replication crisis. Medicine’s crisis is better described as ''potentially'' just as serious, but still less quantified; notable fraud cases and the high cost of irreproducibility keep the issue in the spotlight. Future large-scale replication projects in clinical and pre-clinical medicine will be needed before a definitive comparison is possible.
 == Sources ==
-# https://pubmed.ncbi.nlm.nih.gov/26315443/
+# [https://pubmed.ncbi.nlm.nih.gov/26315443/ Estimating the Reproducibility of Psychological Science – ''Science''] (2015 peer-reviewed replication study)
-# https://unsafescience.substack.com/p/75-of-psychology-claims-are-false
+# [https://unsafescience.substack.com/p/75-of-psychology-claims-are-false ~75 % of Psychology Claims Are False – ''Unsafe Science'' (Substack)] (Opinion / Replication-crisis analysis)
-# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research - The New York Times]
+# [https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html The Long Shadow of Fraud in Alzheimer’s Research – ''The New York Times''] (2025 Opinion / Op-Ed)
-# https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf
+# [https://www.thewikle.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf Revisiting Stereotype Threat: A Reckoning for Social Psychology – Michael Inzlicht] (2024 pre-print PDF; Scholarly essay)
+# [https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time The Staggering Death Toll of Scientific Lies – ''Vox''] (2024 explanatory / analysis article)
 == Question ==
 Is the replication crisis worst in psychology or medicine?