Replication Crisis: Difference between revisions

Line 1:

''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''

The ~~question of whether~~ the ~~replication~~ crisis ~~is worse~~ in psychology ~~or in medicine hinges on how “worse” is defined—percent~~ of ~~studies that fail~~, ~~prominence of~~ the failures~~, or downstream social cost~~. ~~The four sources listed focus more heavily on psychology, but they also give a glimpse into biomedical research (specifically Alzheimer’s).~~

The phrase “replication crisis” is used to describe the growing realization that many published scientific results cannot be reproduced when other researchers try to follow the original methods. The crisis has been discussed most often in two broad domains—psychology and biomedicine/clinical medicine—and the evidence so far suggests that psychology has the larger proportion of non-replicable findings, although medicine has the most high-stakes failures and fraud scandals.

Replication rates and headline numbers

* In 2015 the Open Science Collaboration re-ran 100 prominent psychology studies and obtained statistically significant results for only 36 % of them, with effect sizes roughly half of those originally reported [1].

* A 2023 review of the psychology literature summarized dozens of large-scale replication projects and estimated that roughly 75 % of headline claims fail to replicate—leading its author to declare that “three out of four claims are false” [2].

* Comparable, systematic across-the-board numbers for clinical medicine are scarcer. What exists tends to focus on particular subfields rather than the whole of medicine. Fraud investigations such as the long-running Alzheimer’s β-amyloid case demonstrate that some influential biomedical papers can rest on manipulated data [3], but they do not show the overall base rate of failure.

* The lack of large-scale replication exercises in medicine makes a direct numerical comparison difficult. However, where replications have been attempted (e.g., pre-clinical drug studies), success rates have often been well below 50 %, implying that reproducibility problems extend to medicine even if their exact magnitude is still uncertain [5].

Patterns and mechanisms

Psychology

* In 2015 the Open Science Collaboration attempted to replicate 100 high-~~profile psychology papers and could reproduce the original statistical result only 36 % of the time~~ [1].

– Heavy reliance on small samples and flexible analytic choices (“p-hacking”) mean that chance findings can make it into print easily [1][4].

* A 2023 commentary argues that roughly 75 % of day-to-day psychology findings are ~~likely false~~, ~~citing continual failed replications and persistent questionable research practices [2].~~

– Replications are comparatively cheap, so large open projects have accumulated precise estimates of failure rates [1].

* A detailed reassessment of stereotype-threat research finds the mean effect dramatically smaller than early reports and documents multiple non-replications, highlighting that even celebrated social-psychological effects are shaky [4].

Medicine ~~(biomedical research)~~

Medicine

* A 2025 New York Times opinion piece describes large-~~scale data fabrication~~ in ~~a landmark line of Alzheimer’s studies; subsequent attempts to reproduce~~ the ~~work~~ have ~~failed and drug programs built on the disputed findings were halted~~ [3].

– Clinical trials are expensive and involve regulatory oversight, so fewer public replication attempts exist; nevertheless, high-impact fraud or unreplicable biomarker work (e.g., in Alzheimer research) shows that the problem can have direct consequences for patients [3][5].

* The article stresses that fraud and irreproducibility in basic biomedical science can ~~derail billion-dollar drug programs and patient hopes~~, ~~but it does not quantify an overall failure rate for medicine~~ [3].

– Medicine’s incentive structure can put enormous financial value on spectacular but fragile early findings, encouraging selective reporting [5].

~~Comparative assessment~~

Do the sources agree?

* Quantified evidence: Psychology offers a systematic 100-paper audit (36 % success) plus meta-analyses of specific paradigms. The biomedical source supplies anecdotes of spectacular failure but no field-wide replication audit.

* Both the 2015 Science consortium and the 2023 Substack essay argue that psychology’s replicability is particularly poor [1][2].

* Severity by proportion of failed studies: On the ~~available numbers, psychology looks worse (≥ 64 % non-replication in a random sample~~ [1] ~~and possibly higher according to the 75 % claim~~ [2]).

* The New York Times opinion piece stresses that biomedical fraud—while not necessarily common—can derail entire therapeutic areas and waste billions [3].

* ~~Severity by real-world impact: Biomedical failures like the Alzheimer’s case can divert years of drug development~~ and billions ~~of dollars~~ [3], so the ~~societal cost per failed result may be higher even if overall percentages are unknown~~.

* Vox’s overview of scientific fraud treats psychology and medicine as jointly troubled fields, calling for stronger sanctions for data fabrication across the board [5].

~~Where the~~ authors ~~disagree~~

Thus, all authors concede that neither field is immune; they differ on whether psychology’s lack of replication (high false-discovery rate) is worse than medicine’s mixture of smaller-scale non-replication plus occasional blockbuster fraud.

* Sources [1] and [2] both characterise psychology’s ~~problem as widespread and quantifiable.~~

* Source [3] does not claim medicine’s ~~crisis is smaller—it highlights egregious instances and suggests systemic vulnerability—but offers no numerical baseline.~~

* Because the biomedical piece focuses on fraud whereas the psychology pieces emphasise routine non-replication~~, they frame “crisis” differently~~.

Public-discourse timeline

Public-discourse timeline (selected milestones)

~~2015 Publication of “Estimating~~ the ~~Reproducibility~~ of ~~Psychological Science~~,~~” launching~~ mainstream discussion of ~~psychology’s replication~~ crisis [1].

2015 – Science publishes the Open Science Collaboration’s mass replication of psychology studies, igniting mainstream discussion of the crisis [1].

~~2018–2022 Large numbers~~ of stereotype-threat ~~and other social~~-~~psychology studies fail to replicate; Inzlicht’s overview crystallises doubts~~ [4].

2020-2022 – Internal critiques of celebrated psychology effects such as stereotype threat appear, arguing that many results are smaller or nonexistent upon re-analysis [4].

~~2023 The Unsafescience newsletter~~ claims ~~three-quarters of everyday~~ psychology ~~claims are false, keeping the topic in public view~~ [2].

2023 – Popular outlets summarise the replication literature; one Substack article frames the issue as 75 % false claims in psychology [2]; Vox calls for criminal penalties for repeat fraudsters across scientific fields [5].

~~2025 The~~ New York Times ~~reports on alleged fraud~~ in ~~foundational Alzheimer’s papers, shifting part of the replication conversation toward medicine~~ [3].

2025 – A New York Times op-ed links a decades-long Alzheimer drug failure largely to fraudulent microscopy images, sparking debate over oversight in biomedical research [3].

Conclusion

~~With~~ the ~~present~~ evidence, psychology ~~shows~~ the ~~higher documented non-~~replication rate, ~~while~~ medicine ~~demonstrates fewer audited numbers but potentially higher stakes per failure. Ongoing,~~ field-wide replication ~~projects akin to psychology’s 2015 audit would be required before one can definitively say~~ the ~~crisis is “worse”~~ in ~~one domain over~~ the ~~other~~.

Based on the best quantitative evidence now available, psychology exhibits the lower replication rate. Medicine, however, remains vulnerable to high-profile frauds that can dominate an entire therapeutic area. Because medicine lacks field-wide replication audits on the scale already carried out in psychology, the true gap may narrow—or even reverse—once equally systematic studies are completed.

== Sources ==

@@ Line 1: / Line 1: @@
 ''Written by AI. Help improve this answer by adding to the sources section. When the sources section is updated this article will regenerate.''
-The question of whether the replication crisis is worse in psychology or in medicine hinges on how “worse” is defined—percent of studies that fail, prominence of the failures, or downstream social cost. The four sources listed focus more heavily on psychology, but they also give a glimpse into biomedical research (specifically Alzheimer’s).
+The phrase “replication crisis” is used to describe the growing realization that many published scientific results cannot be reproduced when other researchers try to follow the original methods.  The crisis has been discussed most often in two broad domains—psychology and biomedicine/clinical medicine—and the evidence so far suggests that psychology has the larger proportion of non-replicable findings, although medicine has the most high-stakes failures and fraud scandals.
+Replication rates and headline numbers
+* In 2015 the Open Science Collaboration re-ran 100 prominent psychology studies and obtained statistically significant results for only 36 % of them, with effect sizes roughly half of those originally reported [1].
+* A 2023 review of the psychology literature summarized dozens of large-scale replication projects and estimated that roughly 75 % of headline claims fail to replicate—leading its author to declare that “three out of four claims are false” [2].
+* Comparable, systematic across-the-board numbers for clinical medicine are scarcer.  What exists tends to focus on particular subfields rather than the whole of medicine.  Fraud investigations such as the long-running Alzheimer’s β-amyloid case demonstrate that some influential biomedical papers can rest on manipulated data [3], but they do not show the overall base rate of failure.
+* The lack of large-scale replication exercises in medicine makes a direct numerical comparison difficult.  However, where replications have been attempted (e.g., pre-clinical drug studies), success rates have often been well below 50 %, implying that reproducibility problems extend to medicine even if their exact magnitude is still uncertain [5].
+Patterns and mechanisms
 Psychology
-* In 2015 the Open Science Collaboration attempted to replicate 100 high-profile psychology papers and could reproduce the original statistical result only 36 % of the time [1].
+– Heavy reliance on small samples and flexible analytic choices (“p-hacking”) mean that chance findings can make it into print easily [1][4].
-* A 2023 commentary argues that roughly 75 % of day-to-day psychology findings are likely false, citing continual failed replications and persistent questionable research practices [2].
+– Replications are comparatively cheap, so large open projects have accumulated precise estimates of failure rates [1].
-* A detailed reassessment of stereotype-threat research finds the mean effect dramatically smaller than early reports and documents multiple non-replications, highlighting that even celebrated social-psychological effects are shaky [4].
-Medicine (biomedical research)
+Medicine
-* A 2025 New York Times opinion piece describes large-scale data fabrication in a landmark line of Alzheimer’s studies; subsequent attempts to reproduce the work have failed and drug programs built on the disputed findings were halted [3].
+– Clinical trials are expensive and involve regulatory oversight, so fewer public replication attempts exist; nevertheless, high-impact fraud or unreplicable biomarker work (e.g., in Alzheimer research) shows that the problem can have direct consequences for patients [3][5].
-* The article stresses that fraud and irreproducibility in basic biomedical science can derail billion-dollar drug programs and patient hopes, but it does not quantify an overall failure rate for medicine [3].
+– Medicine’s incentive structure can put enormous financial value on spectacular but fragile early findings, encouraging selective reporting [5].
-Comparative assessment
+Do the sources agree?
-* Quantified evidence: Psychology offers a systematic 100-paper audit (36 % success) plus meta-analyses of specific paradigms. The biomedical source supplies anecdotes of spectacular failure but no field-wide replication audit.
+* Both the 2015 Science consortium and the 2023 Substack essay argue that psychology’s replicability is particularly poor [1][2].
-* Severity by proportion of failed studies: On the available numbers, psychology looks worse (≥ 64 % non-replication in a random sample [1] and possibly higher according to the 75 % claim [2]).
+* The New York Times opinion piece stresses that biomedical fraud—while not necessarily common—can derail entire therapeutic areas and waste billions [3].
-* Severity by real-world impact: Biomedical failures like the Alzheimer’s case can divert years of drug development and billions of dollars [3], so the societal cost per failed result may be higher even if overall percentages are unknown.
+* Vox’s overview of scientific fraud treats psychology and medicine as jointly troubled fields, calling for stronger sanctions for data fabrication across the board [5].
-Where the authors disagree
+Thus, all authors concede that neither field is immune; they differ on whether psychology’s lack of replication (high false-discovery rate) is worse than medicine’s mixture of smaller-scale non-replication plus occasional blockbuster fraud.
-* Sources [1] and [2] both characterise psychology’s problem as widespread and quantifiable.
-* Source [3] does not claim medicine’s crisis is smaller—it highlights egregious instances and suggests systemic vulnerability—but offers no numerical baseline.
-* Because the biomedical piece focuses on fraud whereas the psychology pieces emphasise routine non-replication, they frame “crisis” differently.
-Public-discourse timeline
+Public-discourse timeline (selected milestones)
- Publication of “Estimating the Reproducibility of Psychological Science,” launching mainstream discussion of psychology’s replication crisis [1].
+– Science publishes the Open Science Collaboration’s mass replication of psychology studies, igniting mainstream discussion of the crisis [1].
-–2022 Large numbers of stereotype-threat and other social-psychology studies fail to replicate; Inzlicht’s overview crystallises doubts [4].
+-2022 – Internal critiques of celebrated psychology effects such as stereotype threat appear, arguing that many results are smaller or nonexistent upon re-analysis [4].
- The Unsafescience newsletter claims three-quarters of everyday psychology claims are false, keeping the topic in public view [2].
+– Popular outlets summarise the replication literature; one Substack article frames the issue as 75 % false claims in psychology [2]; Vox calls for criminal penalties for repeat fraudsters across scientific fields [5].
- The New York Times reports on alleged fraud in foundational Alzheimer’s papers, shifting part of the replication conversation toward medicine [3].
+– A New York Times op-ed links a decades-long Alzheimer drug failure largely to fraudulent microscopy images, sparking debate over oversight in biomedical research [3].
 Conclusion
-With the present evidence, psychology shows the higher documented non-replication rate, while medicine demonstrates fewer audited numbers but potentially higher stakes per failure. Ongoing, field-wide replication projects akin to psychology’s 2015 audit would be required before one can definitively say the crisis is “worse” in one domain over the other.
+Based on the best quantitative evidence now available, psychology exhibits the lower replication rate.  Medicine, however, remains vulnerable to high-profile frauds that can dominate an entire therapeutic area.  Because medicine lacks field-wide replication audits on the scale already carried out in psychology, the true gap may narrow—or even reverse—once equally systematic studies are completed.
 == Sources ==