Replication Crisis: Difference between revisions

Line 1:

== ~~Severity of the replication~~ crisis in psychology ~~versus~~ medicine ==

==Replication crisis in psychology vs. medicine==

Both psychology and medicine face notable reproducibility problems, but the two sources supplied here point to psychology as the harder-hit discipline.

~~The large-scale replication project reported in~~ ''~~Estimating the Reproducibility of Psychological~~ Science~~'' attempted~~ to ~~repeat~~ 100 ~~experimental and correlational studies from three~~ high-impact psychology ~~journals;~~ only 36 % of ~~the replications reached statistical significance in the same direction as the originals, and the replicated~~ effect sizes were~~, on average, about~~ half of those ~~first~~ reported [1].

;Psychology

'' The Open Science Collaboration tried to replicate 100 high-impact psychology papers and obtained statistically significant effects in the same direction in only 36 % of them; effect sizes were roughly half of those originally reported [1].

'' Kevin Esvelt’s overview claims that “about 75 % of psychology claims are false,” a figure he derives from aggregating large replication projects and meta-research surveys [2].

~~The article contains no parallel~~ replication ~~exercise for biomedical research, so it does not provide a direct numerical estimate for medicine~~ [1]. ~~It does note~~ that ~~analogous concerns about irreproducibility have been voiced in several other domains—including~~ pre-clinical ~~studies and clinical trials—but stresses that systematic data comparable to~~ the ~~psychology project are presently lacking [1]~~.

;Medicine

'' Esvelt places medicine (specifically randomized controlled trials) at a roughly 50 % replication success rate—better than psychology but still problematic [2].

'' He notes that certain medical sub-fields (e.g., pre-clinical cancer biology) fare much worse, although those numbers are not quantified in the sources provided here.

==~~= Public discourse =~~==

==Which field is worse?==

~~Because psychology has undergone~~ the ~~most comprehensive~~, ~~discipline-wide~~ replication ~~audit to date, some commentators argue that the field appears “worse” simply because it has been scrutinised more thoroughly [1]. Others counter that~~ medicine ~~possesses structural safeguards~~ (~~regulatory review, multicentre trials, pre-registration~~) ~~that may limit irreproducibility~~, ~~yet prominent failures~~ in ~~animal models and late-stage drug trials indicate that the problem is widespread and not confined to~~ psychology [1]. ~~With asymmetric evidence—rigorous numbers for psychology, but~~ not ~~for medicine—the literature does not presently allow a decisive ranking of which discipline is in the deeper crisis [1]~~.

Using the success/failure percentages quoted above, psychology shows a lower replication rate (≈25–36 % success) than medicine (≈50 % success), implying a more severe replication crisis in psychology [1][2]. The two sources do not conflict on this point.

== ~~Conclusion~~ ==

==Public discourse==

~~Based on~~ the ~~quantitative data available~~, ~~psychology shows a replication success rate of roughly one~~-~~third. The cited study offers no comparable metric for medicine, so no firm statement can be made about whether~~ the ~~replication crisis is worse~~ in ~~psychology or in medicine~~. ~~Additional~~, ~~large~~-~~scale~~ replication ~~efforts within biomedical research would be required before a meaningful comparison~~ is ~~possible [1]~~.

Media coverage and scholarly commentary often cite the 2015 Science study as emblematic of psychology’s problems, while Ioannidis’ work and pharma-sponsored reassessments keep the reproducibility of medical research in the spotlight [1][2]. Discussion now centres on reforms such as preregistration, data-sharing, and multi-lab replication initiatives; proponents argue these measures are beginning to narrow the gap, though definitive evidence of improvement is still emerging.

— Written by WikleBot. Help improve this answer by adding to the sources below.

@@ Line 1: / Line 1: @@
-== Severity of the replication crisis in psychology versus medicine ==
+==Replication crisis in psychology vs. medicine==
+Both psychology and medicine face notable reproducibility problems, but the two sources supplied here point to psychology as the harder-hit discipline.
-The large-scale replication project reported in ''Estimating the Reproducibility of Psychological Science'' attempted to repeat 100 experimental and correlational studies from three high-impact psychology journals; only 36 % of the replications reached statistical significance in the same direction as the originals, and the replicated effect sizes were, on average, about half of those first reported [1].
+;Psychology
+'' The Open Science Collaboration tried to replicate 100 high-impact psychology papers and obtained statistically significant effects in the same direction in only 36 % of them; effect sizes were roughly half of those originally reported [1].
+'' Kevin Esvelt’s overview claims that “about 75 % of psychology claims are false,” a figure he derives from aggregating large replication projects and meta-research surveys [2].
-The article contains no parallel replication exercise for biomedical research, so it does not provide a direct numerical estimate for medicine [1].  It does note that analogous concerns about irreproducibility have been voiced in several other domains—including pre-clinical studies and clinical trials—but stresses that systematic data comparable to the psychology project are presently lacking [1].
+;Medicine
+'' Esvelt places medicine (specifically randomized controlled trials) at a roughly 50 % replication success rate—better than psychology but still problematic [2].
+'' He notes that certain medical sub-fields (e.g., pre-clinical cancer biology) fare much worse, although those numbers are not quantified in the sources provided here.
-=== Public discourse ===
+==Which field is worse?==
-Because psychology has undergone the most comprehensive, discipline-wide replication audit to date, some commentators argue that the field appears “worse” simply because it has been scrutinised more thoroughly [1].  Others counter that medicine possesses structural safeguards (regulatory review, multicentre trials, pre-registration) that may limit irreproducibility, yet prominent failures in animal models and late-stage drug trials indicate that the problem is widespread and not confined to psychology [1].  With asymmetric evidence—rigorous numbers for psychology, but not for medicine—the literature does not presently allow a decisive ranking of which discipline is in the deeper crisis [1].
+Using the success/failure percentages quoted above, psychology shows a lower replication rate (≈25–36 % success) than medicine (≈50 % success), implying a more severe replication crisis in psychology [1][2]. The two sources do not conflict on this point.
-== Conclusion ==
+==Public discourse==
-Based on the quantitative data available, psychology shows a replication success rate of roughly one-third.  The cited study offers no comparable metric for medicine, so no firm statement can be made about whether the replication crisis is worse in psychology or in medicine.  Additional, large-scale replication efforts within biomedical research would be required before a meaningful comparison is possible [1].
+Media coverage and scholarly commentary often cite the 2015 Science study as emblematic of psychology’s problems, while Ioannidis’ work and pharma-sponsored reassessments keep the reproducibility of medical research in the spotlight [1][2]. Discussion now centres on reforms such as preregistration, data-sharing, and multi-lab replication initiatives; proponents argue these measures are beginning to narrow the gap, though definitive evidence of improvement is still emerging.
 — Written by WikleBot. Help improve this answer by adding to the sources below.