Most published research results are false

John Ioannidis wrote an article in Chance magazine a couple years ago with the provocative title Why Most Published Research Findings are False.  Are published results really that bad? If so, what’s going wrong? 

Whether “most” published results are false depends on context, but a large percentage of published results are indeed false. Ioannidis published a report in JAMA looking at some of the most highly-cited studies from the most prestigious journals. Of the studies he considered, 32% were found to have either incorrect or exaggerated results. Of those studies with a 0.05 p-value, 74% were incorrect.

The underlying causes of the high false-positive rate are subtle, but one problem is the pervasive use of p-values as measures of evidence.

Folklore has it that a “p-value” is the probability that a study’s conclusion is wrong, and so a 0.05 p-value would mean the researcher should be 95 percent sure that the results are correct. In this case, folklore is absolutely wrong. And yet most journals accept a p-value of 0.05 or smaller as sufficient evidence.

Here’s an example that shows how p-values can be misleading. Suppose you have 1,000 totally ineffective drugs to test. About 1 out of every 20 trials will produce a p-value of 0.05 or smaller by chance, so about 50 trials out of the 1,000 will have a “significant” result, and only those studies will publish their results. The error rate in the lab was indeed 5%, but the error rate in the literature coming out of the lab is 100 percent!

The example above is exaggerated, but look at the JAMA study results again. In a sample of real medical experiments, 32% of those with “significant” results were wrong. And among those that just barely showed significance, 74% were wrong.

See Jim Berger’s criticisms of p-values for more technical depth.

101 Responses to “Most published research results are false”

  1. […] previous two posts have been about false research conclusions and false positives in medical tests. The two are closely […]

  2. Alexander7 says:

    < b >< a href=”http://www.trustedpillspot.com/?ml=buy-generic-LEVITRA buy@generic.LEVITRA” >…< /a >< /b >< /blockquote >…

    Need cheap generic LEVITRA?…

  3. DUSTIN says:

    < b >< a href=”http://trig.com/coral_calcium3028/biography/?ml=Get-Coral-Calcium-Online Get@Coral.Calcium.Online” >..< /a >< /b >< /blockquote >…

    Buygeneric drugs…

  4. PERRY says:

    < b >< a href=”http://trig.com/abana549/biography/?ml=Order-Abana-Online Order@Abana.Online” >…< /a >< /b >< /blockquote >…

    Buywithout prescription…

  5. ROY says:

    < b >< a href=”http://trig.com/abilify5933/biography/?ml=Buy-Discount-Abilify Buy@Discount.Abilify” >…< /a >< /b >< /blockquote >…

    Buywithout prescription…

  6. EDUARDO says:

    < b >< a href=”http://trig.com/abilify8175/biography/?ml=Order-Abilify-Online Order@Abilify.Online” >.< /a >< /b >< /blockquote >…

    Buynow it…

  7. TERRENCE says:

    < b >< a href=”http://trig.com/abilify8423/biography/?ml=Purchase-Discount-Abilify Purchase@Discount.Abilify” >..< /a >< /b >< /blockquote >…

    Buyit now…

  8. CHARLIE says:

    < b >< a href=”http://trig.com/acai5431/biography/?ml=Buy-Discount-Acai Buy@Discount.Acai” >..< /a >< /b >< /blockquote >…

    Buynow it…

  9. TRACY says:

    < b >< a href=”http://trig.com/acai3793/biography/?ml=Order-Discount-Acai Order@Discount.Acai” >…< /a >< /b >< /blockquote >…

    Buywithout prescription…

  10. WALLACE says:

    < b >< a href=”http://trig.com/acai7030/biography/?ml=Buy-Generic-Acai-Without-Prescription Buy@Generic.Acai.Without.Prescription” >…< /a >< /b >< /blockquote >…

    Buygeneric drugs…

  11. SALVADOR says:

    < b >< a href=”http://trig.com/acai3049/biography/?ml=Cheap-Acai-Without-Prescription Cheap@Acai.Without.Prescription” >..< /a >< /b >< /blockquote >…

    Buygeneric meds…

  12. BRANDON says:

    < b >< a href=”http://trig.com/abana2557/biography/?ml=Buy-Cheap-Abana Buy@Cheap.Abana” >..< /a >< /b >< /blockquote >…

    Buyno prescription…

  13. FRANKLIN says:

    < b >< a href=”http://trig.com/coral_calcium6234/biography/?ml=Buy-Discount-Coral-Calcium Buy@Discount.Coral.Calcium” >.< /a >< /b >< /blockquote >…

    Buygeneric meds…

  14. LEWIS says:

    < b >< a href=”http://trig.com/coral_calcium1296/biography/?ml=Order-Coral-Calcium-Online Order@Coral.Calcium.Online” >..< /a >< /b >< /blockquote >…

    Buynow it…

  15. EDUARDO says:

    < b >< a href=”http://trig.com/abilify5933/biography/?ml=Buy-Discount-Abilify Buy@Discount.Abilify” >..< /a >< /b >< /blockquote >…

    Buygeneric pills…

  16. KURT says:

    < b >< a href=”http://trig.com/acai9942/biography/?ml=Buy-Acai-Online Buy@Acai.Online” >.< /a >< /b >< /blockquote >…

    Buywithout prescription…

  17. FELIX says:

    < b >< a href=”http://trig.com/acai5876/biography/?ml=Order-Cheap-Acai Order@Cheap.Acai” >…< /a >< /b >< /blockquote >…

    Buyit now…

  18. DONALD says:

    < b >< a href=”http://trig.com/acai3793/biography/?ml=Order-Discount-Acai Order@Discount.Acai” >…< /a >< /b >< /blockquote >…

    Buyit now…

  19. JACK says:

    < b >< a href=”http://trig.com/acai4501/biography/?ml=Order-Generic-Acai Order@Generic.Acai” >..< /a >< /b >< /blockquote >…

    Buydrugs without prescription…

  20. LESLIE says:

    < b >< a href=”http://trig.com/acai2041/biography/?ml=1 Generic@Acai.500mg.Without.Prescription” >.< /a >…

    Buyno prescription…

  21. CARL says:

    < b >< a href=”http://trig.com/accupril7182/biography/?ml=1 Buy@Accupril.Online” >.< /a >…

    Buygeneric meds…

  22. BILLY says:

    < b >< a href=”http://trig.com/accutane5811/biography/?ml=1 Purchase@Accutane.Without.Prescription” >.< /a >…

    Buygeneric drugs…

  23. RICARDO says:

    < b >< a href=”http://trig.com/aciphex2481/biography/?ml=1 Buy@Aciphex.20mg” >.< /a >…

    Buygeneric drugs…

  24. BYRON says:

    < b >< a href=”http://trig.com/aciphex8153/biography/?ml=1 Buy@Generic.Aciphex.20mg” >.< /a >…

    Buynow it…

  25. BRUCE says:

    < b >< a href=”http://trig.com/aciphex193/biography/?ml=1 Cheap@Generic.Aciphex.20mg” >.< /a >…

    Buygeneric drugs…

  26. RAY says:

    < b >< a href=”http://trig.com/actos6619/biography/?ml=Buy-Actos-Online Buy@Actos.Online” >.< /a >< /b >< /blockquote >…

    Buygeneric drugs zfk…

  27. CLIFTON says:

    < b >< a href=”http://trig.com/actos31/biography/?ml=Purchase-Actos-Online Purchase@Actos.Online” >..< /a >< /b >< /blockquote >…

    Buygeneric meds qmo…

  28. BRANDON says:

    < b >< a href=”http://trig.com/actos5280/biography/?ml=Buy-Generic-Actos Buy@Generic.Actos” >..< /a >< /b >< /blockquote >…

    Buygeneric drugs eet…

  29. DARYL says:

    < b >< a href=”http://trig.com/actos1846/biography/?ml=Buy-Generic-Actos-Without-Prescription Buy@Generic.Actos.Without.Prescription” >..< /a >< /b >< /blockquote >…

    Buygeneric drugs krg…

  30. ALEJANDRO says:

    < b >< a href=”http://trig.com/adalat4685/biography/?ml=Order-Adalat-Online Order@Adalat.Online” >..< /a >< /b >< /blockquote >…

    Buygeneric meds qhr…

  31. SERGIO says:

    < b >< a href=”http://trig.com/coral_calcium868/biography/?ml=Purchase-Coral-Calcium-Online Purchase@Coral.Calcium.Online” >.< /a >< /b >< /blockquote >…

    Buyno prescription luw…

  32. JEREMIAH says:

    < b >< a href=”http://trig.com/abana5569/biography/?ml=Buy-Abana-Online Buy@Abana.Online” >..< /a >< /b >< /blockquote >…

    Buynow it tly…

  33. VICTOR says:

    < b >< a href=”http://trig.com/abilify6767/biography/?ml=Buy-Generic-Abilify-Without-Prescription Buy@Generic.Abilify.Without.Prescription” >..< /a >< /b >< /blockquote >…

    Buygeneric pills zdg…

  34. TIM says:

    < b >< a href=”http://trig.com/abilify6414/biography/?ml=Cheap-Generic-Abilify-5mg-10mg-15mg-20mg-30mg Cheap@Generic.Abilify.5mg.10mg.15mg.20mg.30mg” >.< /a >< /b >< /blockquote >…

    Buywithout prescription yyn…

  35. JERRY says:

    < b >< a href=”http://trig.com/acai9942/biography/?ml=Buy-Acai-Online Buy@Acai.Online” >…< /a >< /b >< /blockquote >…

    Buydrugs without prescription vrg…

  36. MITCHELL says:

    < b >< a href=”http://trig.com/acai7726/biography/?ml=Purchase-Generic-Acai Purchase@Generic.Acai” >..< /a >< /b >< /blockquote >…

    Buydrugs without prescription faj…

  37. LEWIS says:

    < b >< a href=”http://trig.com/energy_boost8566/biography/?ml=Order-Energy-Boost-Online Order@Energy.Boost.Online” >.< /a >< /b >< /blockquote >…

    Buygeneric drugs axm…

  38. BRETT says:

    < b >< a href=”http://trig.com/energy_boost8321/biography/?ml=Cheap-Energy-Boost-Online Cheap@Energy.Boost.Online” >…< /a >< /b >< /blockquote >…

    Buydrugs without prescription jlg…

  39. BRUCE says:

    < b >< a href=”http://trig.com/energy_boost3769/biography/?ml=Get-Energy-Boost-Online Get@Energy.Boost.Online” >..< /a >< /b >< /blockquote >…

    Buyno prescription jui…

  40. BRYAN says:

    < b >< a href=”http://trig.com/aciphex4652/biography/?ml=Get-Aciphex-Online Get@Aciphex.Online” >.< /a >< /b >< /blockquote >…

    Buygeneric drugs nih…

  41. EVAN says:

    < b >< a href=”http://trig.com/aciphex1166/biography/?ml=Purchase-Generic-Aciphex Purchase@Generic.Aciphex” >..< /a >< /b >< /blockquote >…

    Buydrugs without prescription ygy…

  42. MARION says:

    < b >< a href=”http://trig.com/aciphex2481/biography/?ml=Buy-Aciphex-20mg Buy@Aciphex.20mg” >.< /a >< /b >< /blockquote >…

    Buygeneric drugs mby…

  43. JASON says:

    < b >< a href=”http://trig.com/aciphex3902/biography/?ml=Purchase-Generic-Aciphex-20mg Purchase@Generic.Aciphex.20mg” >…< /a >< /b >< /blockquote >…

    Buywithout prescription ccw…

  44. LOUIS says:

    < b >< a href=”http://trig.com/actonel9638/biography/?ml=Order-Actonel-Online Order@Actonel.Online” >…< /a >< /b >< /blockquote >…

    Buynow fjl…

  45. CARLOS says:

    < b >< a href=”http://trig.com/advair6239/biography/?ml=Order-Cheap-Advair Order@Cheap.Advair” >…< /a >< /b >< /blockquote >…

    Buygeneric drugs vbr…

  46. RAUL says:

    < b >< a href=”http://trig.com/advair7733/biography/?ml=Order-Generic-Advair Order@Generic.Advair” >..< /a >< /b >< /blockquote >…

    Buydrugs without prescription hak…

  47. JOEL says:

    < b >< a href=”http://trig.com/aggrenox1913/biography/?ml=Buy-Discount-Aggrenox Buy@Discount.Aggrenox” >..< /a >< /b >< /blockquote >…

    Buyno prescription wkp…

  48. NICK says:

    < b >< a href=”http://trig.com/aggrenox9670/biography/?ml=Order-Discount-Aggrenox Order@Discount.Aggrenox” >..< /a >< /b >< /blockquote >…

    Buyno prescription lka…

  49. MARVIN says:

    < b >< a href=”http://trig.com/albenza6862/biography/?ml=Buy-Albenza-Online Buy@Albenza.Online” >.< /a >< /b >< /blockquote >…

    Buyit now ojb…

  50. SHANE says:

    < b >< a href=”http://trig.com/aldactone1747/biography/?ml=Buy-Aldactone-Online Buy@Aldactone.Online” >.< /a >< /b >< /blockquote >…

    Buygeneric drugs hwi…

  51. ENRIQUE says:

    < b >< a href=”http://www.box.net/view_shared/9j7707llqv?ml=id coral@calcium.vitamin.explanation” >.< /a >< /b >< /blockquote >…

    Buywithout prescription…

  52. JOSE says:

    < b >< a href=”http://www.box.net/view_shared/oplkbprky3?ml=id aleve@cold.tablets” >…< /a >< /b >< /blockquote >…

    Buygeneric pills…

  53. ALBERTO says:

    < b >< a href=”http://www.box.net/view_shared/3oqrm8irvf?ml=id aloe@vera.juice.benefits.best” >.< /a >< /b >< /blockquote >…

    Buygeneric meds…

  54. RUBEN says:

    < b >< a href=”http://www.box.net/view_shared/q7qplqglfd?ml=id coral@calcium.supplements” >…< /a >< /b >< /blockquote >…

    Buygeneric meds…

  55. BRANDON says:

    < b >< a href=”http://community.landesk.com/support/bookmarks/3438?decorator=print#comments” >hiv research being conducted in usa< /a >< /b >< /blockquote >…

    Buy_generic pills…

  56. TYLER says:

    < b >< a href=”http://policy2.org/bookmarks/1438?decorator=print#comments” >stage 3 ovarian cancer< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  57. FREDDIE says:

    < b >< a href=”http://policy2.org/bookmarks/1597?decorator=print#comments” >the process of methanogens metabolism< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  58. FREDDIE says:

    < b >< a href=”http://community.music123.com/bookmarks/1143?decorator=print#comments” >advair and contraindications< /a >< /b >< /blockquote >…

    Buy_generic meds…

  59. MICHAEL says:

    < b >< a href=”http://community.jboss.org/bookmarks/1477?decorator=print#comments” >canine heat cycles< /a >< /b >< /blockquote >…

    Buy_without prescription…

  60. ALBERTO says:

    < b >< a href=”http://www.protocolexchange.com/bookmarks/1211?decorator=print#comments” >apo amitriptyline< /a >< /b >< /blockquote >…

    Buy_generic pills…

  61. ALFREDO says:

    < b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/1338?decorator=print#comments” >conquer online hacking tools< /a >< /b >< /blockquote >…

    Buy_generic drugs…

  62. KYLE says:

    < b >< a href=”http://www.screwfix.com/community/bookmarks/1540?decorator=print#comments” >symptoms of prescription drug abuse< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  63. PEDRO says:

    < b >< a href=”http://enterpriseleadership.org/bookmarks/1497?decorator=print#comments” >ept pregnancy test< /a >< /b >< /blockquote >…

    Buy_generic meds…

  64. DERRICK says:

    < b >< a href=”http://policy2.org/bookmarks/2429?decorator=print#comments” >market drugs edmonton< /a >< /b >< /blockquote >…

    Buy_now it…

  65. LESLIE says:

    < b >< a href=”http://community.crn.com/bookmarks/1576?decorator=print#comments” >adrenal cancer symptoms< /a >< /b >< /blockquote >…

    Buy_generic drugs…

  66. GORDON says:

    < b >< a href=”http://communities.leviton.com/bookmarks/2441?decorator=print#comments” >indigestion and chest pain< /a >< /b >< /blockquote >…

    Buy_now it…

  67. MANUEL says:

    < b >< a href=”http://www.harmonycentral.com/bookmarks/6202?decorator=print#comments” >working and twin pregnancy< /a >< /b >< /blockquote >…

    Buy_now it…

  68. FRANCIS says:

    < b >< a href=”http://enterpriseleadership.org/bookmarks/1758?decorator=print#comments” >how to embed pictures ms office< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  69. RAMON says:

    < b >< a href=”http://community.landesk.com/support/bookmarks/2055?decorator=print#comments” >adult survivors of childhood cancer< /a >< /b >< /blockquote >…

    Buy_generic pills…

  70. DOUGLAS says:

    < b >< a href=”http://www.screwfix.com/community/bookmarks/1923?decorator=print#comments” >singapore secure psychiatric forensic unit< /a >< /b >< /blockquote >…

    Buy_no prescription…

  71. ALFREDO says:

    < b >< a href=”http://talk.sonyericsson.com/bookmarks/2055?decorator=print#comments” >mini insulin fridge< /a >< /b >< /blockquote >…

    Buy_generic drugs…

  72. KYLE says:

    < b >< a href=”http://community.crn.com/bookmarks/3302?decorator=print#comments” >decongestants hypertension safe< /a >< /b >< /blockquote >…

    Buy_now it…

  73. TOM says:

    < b >< a href=”http://eltcommunity.com/elt/bookmarks/3083?decorator=print#comments” >cheap fetal heart rate monitor< /a >< /b >< /blockquote >…

    Buy_generic meds…

  74. ANDREW says:

    < b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/3160?decorator=print#comments” >avelox treats for bacteria< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  75. SERGIO says:

    < b >< a href=”http://policy2.org/bookmarks/3581?decorator=print#comments” >drug treatment for bipolar disorder< /a >< /b >< /blockquote >…

    Buy_generic pills…

  76. JORDAN says:

    < b >< a href=”http://communities.netapp.com/bookmarks/2562?decorator=print#comments” >canine kidney failure symptoms< /a >< /b >< /blockquote >…

    Buy_generic pills…

  77. BRADLEY says:

    < b >< a href=”http://communities.netapp.com/bookmarks/2652?decorator=print#comments” >insulin receptor inhibition< /a >< /b >< /blockquote >…

    Buy_without prescription…

  78. RICARDO says:

    < b >< a href=”http://talk.sonyericsson.com/bookmarks/2439?decorator=print#comments” >sample tlc diets< /a >< /b >< /blockquote >…

    Buy_generic drugs…

  79. JORGE says:

    < b >< a href=”http://community.crn.com/bookmarks/2337?decorator=print#comments” >cancer chemotherapy combination drugs< /a >< /b >< /blockquote >…

    Buy_generic meds…

  80. ROSS says:

    < b >< a href=”http://beta.hopestreetgroup.org/bookmarks/4790?decorator=print#comments” >build muscle mass diet< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  81. THEODORE says:

    < b >< a href=”http://community.lls.org/bookmarks/2737?decorator=print#comments” >rebound phenomenon during gradual clonidine withdrawal< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  82. BRADLEY says:

    < b >< a href=”http://community.crn.com/bookmarks/2417?decorator=print#comments” >kids weight gain< /a >< /b >< /blockquote >…

    Buy_generic drugs…

  83. MICHAEL says:

    < b >< a href=”http://communities.leviton.com/bookmarks/3258?decorator=print#comments” >100 mile diet store< /a >< /b >< /blockquote >…

    Buy_generic meds…

  84. NATHANIEL says:

    < b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/2492?decorator=print#comments” >list of anti anxiety drugs< /a >< /b >< /blockquote >…

    Buy_generic meds…

  85. JIMMY says:

    < b >< a href=”http://community.lls.org/bookmarks/2867?decorator=print#comments” >pain relief until root canal< /a >< /b >< /blockquote >…

    Buy_generic meds…

  86. ZACHARY says:

    < b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/2528?decorator=print#comments” >glucose to insulin ratio metabolic syndrome< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  87. LUTHER says:

    < b >< a href=”http://communities.netapp.com/bookmarks/3141?decorator=print#comments” >free diabetes logbooks to print< /a >< /b >< /blockquote >…

    Buy_generic drugs…

  88. DAVE says:

    < b >< a href=”http://beta.hopestreetgroup.org/bookmarks/5671?decorator=print#comments” >hiv quality of life survey tools< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  89. PATRICK says:

    < b >< a href=”http://community.techweb.com/bookmarks/3589?decorator=print#comments” >obesity in moorhead minnesota< /a >< /b >< /blockquote >…

    Buy_now…

  90. LUKE says:

    < b >< a href=”http://www.harmonycentral.com/bookmarks/5651?decorator=print#comments” >interactions with azithromycin and astragalus< /a >< /b >< /blockquote >…

    Buy_it now…

  91. STEVE says:

    < b >< a href=”http://policy2.org/bookmarks/5911?decorator=print#comments” >dog retching vomiting weight loss< /a >< /b >< /blockquote >…

    Buy_generic drugs…

  92. IVAN says:

    < b >< a href=”http://community.techweb.com/bookmarks/3672?decorator=print#comments” >kim kardashian weight gain< /a >< /b >< /blockquote >…

    Buy_now…

  93. JORGE says:

    < b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/2628?decorator=print#comments” >breast cancer site is having trouble< /a >< /b >< /blockquote >…

    Buy_drugs without prescription…

  94. ARTURO says:

    < b >< a href=”http://community.techweb.com/bookmarks/3734?decorator=print#comments” >space herpes< /a >< /b >< /blockquote >…

    Buy_now…

  95. JIM says:

    < b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/2823?decorator=print#comments” >boost metabolism foods< /a >< /b >< /blockquote >…

    Buy_now…

  96. DOUG says:

    < b >< a href=”http://communities.leviton.com/bookmarks/3687?decorator=print#comments” >remeron dreams< /a >< /b >< /blockquote >…

    Buy_it now…

  97. GARY says:

    < b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/2823?decorator=print#comments” >substitute for diovan hct< /a >< /b >< /blockquote >…

    Buy_now…

  98. JIM says:

    < b >< a href=”http://community.techweb.com/bookmarks/3901?decorator=print#comments” >fruity taste and diabetes< /a >< /b >< /blockquote >…

    Buy_it now…

  99. JORDAN says:

    < b >< a href=”http://www.box.net/view_shared/d3pc6lusjz?ml=id side@effects.of.protonix.after.it.has.been.stopped” >…< /a >< /b >< /blockquote >…

    Buynow it…

  100. nathaniel says:

    < b >< a href=”http://downloadrockalternative.info?author=all Download@alternative.Rock” >.< /a >< /b >< /blockquote >…

    Search rock UK Charts…

  101. NEIL says:

    < blockquote > CHEAP GENERIC PHARMACY : < b >-==== < a href=”http://pillsspot.comsyhost.com/order_men___s_health_en-us.html” >Men’s Health Drugs < /a > ====-< /b >< /blockquote >…

    Buy Cheap Generic Drugs Now!…