John Ioannidis wrote an article in Chance magazine a couple years ago with the provocative title Why Most Published Research Findings are False. Are published results really that bad? If so, what’s going wrong?
Whether “most” published results are false depends on context, but a large percentage of published results are indeed false. Ioannidis published a report in JAMA looking at some of the most highly-cited studies from the most prestigious journals. Of the studies he considered, 32% were found to have either incorrect or exaggerated results. Of those studies with a 0.05 p-value, 74% were incorrect.
The underlying causes of the high false-positive rate are subtle, but one problem is the pervasive use of p-values as measures of evidence.
Folklore has it that a “p-value” is the probability that a study’s conclusion is wrong, and so a 0.05 p-value would mean the researcher should be 95 percent sure that the results are correct. In this case, folklore is absolutely wrong. And yet most journals accept a p-value of 0.05 or smaller as sufficient evidence.
Here’s an example that shows how p-values can be misleading. Suppose you have 1,000 totally ineffective drugs to test. About 1 out of every 20 trials will produce a p-value of 0.05 or smaller by chance, so about 50 trials out of the 1,000 will have a “significant” result, and only those studies will publish their results. The error rate in the lab was indeed 5%, but the error rate in the literature coming out of the lab is 100 percent!
The example above is exaggerated, but look at the JAMA study results again. In a sample of real medical experiments, 32% of those with “significant” results were wrong. And among those that just barely showed significance, 74% were wrong.
See Jim Berger’s criticisms of p-values for more technical depth.
2 May 2008 at 18:39
[…] previous two posts have been about false research conclusions and false positives in medical tests. The two are closely […]
17 July 2011 at 11:53
< b >< a href=”http://www.trustedpillspot.com/?ml=buy-generic-LEVITRA buy@generic.LEVITRA” >…< /a >< /b >< /blockquote >…
Need cheap generic LEVITRA?…
29 July 2011 at 03:58
< b >< a href=”http://trig.com/coral_calcium3028/biography/?ml=Get-Coral-Calcium-Online Get@Coral.Calcium.Online” >..< /a >< /b >< /blockquote >…
Buygeneric drugs…
29 July 2011 at 05:37
< b >< a href=”http://trig.com/abana549/biography/?ml=Order-Abana-Online Order@Abana.Online” >…< /a >< /b >< /blockquote >…
Buywithout prescription…
29 July 2011 at 12:06
< b >< a href=”http://trig.com/abilify5933/biography/?ml=Buy-Discount-Abilify Buy@Discount.Abilify” >…< /a >< /b >< /blockquote >…
Buywithout prescription…
29 July 2011 at 12:31
< b >< a href=”http://trig.com/abilify8175/biography/?ml=Order-Abilify-Online Order@Abilify.Online” >.< /a >< /b >< /blockquote >…
Buynow it…
29 July 2011 at 14:32
< b >< a href=”http://trig.com/abilify8423/biography/?ml=Purchase-Discount-Abilify Purchase@Discount.Abilify” >..< /a >< /b >< /blockquote >…
Buyit now…
30 July 2011 at 05:27
< b >< a href=”http://trig.com/acai5431/biography/?ml=Buy-Discount-Acai Buy@Discount.Acai” >..< /a >< /b >< /blockquote >…
Buynow it…
30 July 2011 at 10:35
< b >< a href=”http://trig.com/acai3793/biography/?ml=Order-Discount-Acai Order@Discount.Acai” >…< /a >< /b >< /blockquote >…
Buywithout prescription…
30 July 2011 at 23:39
< b >< a href=”http://trig.com/acai7030/biography/?ml=Buy-Generic-Acai-Without-Prescription Buy@Generic.Acai.Without.Prescription” >…< /a >< /b >< /blockquote >…
Buygeneric drugs…
31 July 2011 at 09:00
< b >< a href=”http://trig.com/acai3049/biography/?ml=Cheap-Acai-Without-Prescription Cheap@Acai.Without.Prescription” >..< /a >< /b >< /blockquote >…
Buygeneric meds…
1 August 2011 at 04:26
< b >< a href=”http://trig.com/abana2557/biography/?ml=Buy-Cheap-Abana Buy@Cheap.Abana” >..< /a >< /b >< /blockquote >…
Buyno prescription…
2 August 2011 at 01:16
< b >< a href=”http://trig.com/coral_calcium6234/biography/?ml=Buy-Discount-Coral-Calcium Buy@Discount.Coral.Calcium” >.< /a >< /b >< /blockquote >…
Buygeneric meds…
2 August 2011 at 02:03
< b >< a href=”http://trig.com/coral_calcium1296/biography/?ml=Order-Coral-Calcium-Online Order@Coral.Calcium.Online” >..< /a >< /b >< /blockquote >…
Buynow it…
2 August 2011 at 22:44
< b >< a href=”http://trig.com/abilify5933/biography/?ml=Buy-Discount-Abilify Buy@Discount.Abilify” >..< /a >< /b >< /blockquote >…
Buygeneric pills…
3 August 2011 at 13:01
< b >< a href=”http://trig.com/acai9942/biography/?ml=Buy-Acai-Online Buy@Acai.Online” >.< /a >< /b >< /blockquote >…
Buywithout prescription…
3 August 2011 at 19:02
< b >< a href=”http://trig.com/acai5876/biography/?ml=Order-Cheap-Acai Order@Cheap.Acai” >…< /a >< /b >< /blockquote >…
Buyit now…
3 August 2011 at 21:00
< b >< a href=”http://trig.com/acai3793/biography/?ml=Order-Discount-Acai Order@Discount.Acai” >…< /a >< /b >< /blockquote >…
Buyit now…
4 August 2011 at 15:41
< b >< a href=”http://trig.com/acai4501/biography/?ml=Order-Generic-Acai Order@Generic.Acai” >..< /a >< /b >< /blockquote >…
Buydrugs without prescription…
4 August 2011 at 23:15
< b >< a href=”http://trig.com/acai2041/biography/?ml=1 Generic@Acai.500mg.Without.Prescription” >.< /a >…
Buyno prescription…
5 August 2011 at 11:38
< b >< a href=”http://trig.com/accupril7182/biography/?ml=1 Buy@Accupril.Online” >.< /a >…
Buygeneric meds…
6 August 2011 at 17:40
< b >< a href=”http://trig.com/accutane5811/biography/?ml=1 Purchase@Accutane.Without.Prescription” >.< /a >…
Buygeneric drugs…
8 August 2011 at 13:17
< b >< a href=”http://trig.com/aciphex2481/biography/?ml=1 Buy@Aciphex.20mg” >.< /a >…
Buygeneric drugs…
8 August 2011 at 14:22
< b >< a href=”http://trig.com/aciphex8153/biography/?ml=1 Buy@Generic.Aciphex.20mg” >.< /a >…
Buynow it…
8 August 2011 at 16:27
< b >< a href=”http://trig.com/aciphex193/biography/?ml=1 Cheap@Generic.Aciphex.20mg” >.< /a >…
Buygeneric drugs…
11 August 2011 at 03:09
< b >< a href=”http://trig.com/actos6619/biography/?ml=Buy-Actos-Online Buy@Actos.Online” >.< /a >< /b >< /blockquote >…
Buygeneric drugs zfk…
11 August 2011 at 06:33
< b >< a href=”http://trig.com/actos31/biography/?ml=Purchase-Actos-Online Purchase@Actos.Online” >..< /a >< /b >< /blockquote >…
Buygeneric meds qmo…
11 August 2011 at 12:31
< b >< a href=”http://trig.com/actos5280/biography/?ml=Buy-Generic-Actos Buy@Generic.Actos” >..< /a >< /b >< /blockquote >…
Buygeneric drugs eet…
11 August 2011 at 13:34
< b >< a href=”http://trig.com/actos1846/biography/?ml=Buy-Generic-Actos-Without-Prescription Buy@Generic.Actos.Without.Prescription” >..< /a >< /b >< /blockquote >…
Buygeneric drugs krg…
12 August 2011 at 12:27
< b >< a href=”http://trig.com/adalat4685/biography/?ml=Order-Adalat-Online Order@Adalat.Online” >..< /a >< /b >< /blockquote >…
Buygeneric meds qhr…
12 August 2011 at 20:23
< b >< a href=”http://trig.com/coral_calcium868/biography/?ml=Purchase-Coral-Calcium-Online Purchase@Coral.Calcium.Online” >.< /a >< /b >< /blockquote >…
Buyno prescription luw…
12 August 2011 at 23:10
< b >< a href=”http://trig.com/abana5569/biography/?ml=Buy-Abana-Online Buy@Abana.Online” >..< /a >< /b >< /blockquote >…
Buynow it tly…
13 August 2011 at 11:59
< b >< a href=”http://trig.com/abilify6767/biography/?ml=Buy-Generic-Abilify-Without-Prescription Buy@Generic.Abilify.Without.Prescription” >..< /a >< /b >< /blockquote >…
Buygeneric pills zdg…
13 August 2011 at 20:21
< b >< a href=”http://trig.com/abilify6414/biography/?ml=Cheap-Generic-Abilify-5mg-10mg-15mg-20mg-30mg Cheap@Generic.Abilify.5mg.10mg.15mg.20mg.30mg” >.< /a >< /b >< /blockquote >…
Buywithout prescription yyn…
13 August 2011 at 23:54
< b >< a href=”http://trig.com/acai9942/biography/?ml=Buy-Acai-Online Buy@Acai.Online” >…< /a >< /b >< /blockquote >…
Buydrugs without prescription vrg…
14 August 2011 at 17:33
< b >< a href=”http://trig.com/acai7726/biography/?ml=Purchase-Generic-Acai Purchase@Generic.Acai” >..< /a >< /b >< /blockquote >…
Buydrugs without prescription faj…
15 August 2011 at 10:21
< b >< a href=”http://trig.com/energy_boost8566/biography/?ml=Order-Energy-Boost-Online Order@Energy.Boost.Online” >.< /a >< /b >< /blockquote >…
Buygeneric drugs axm…
15 August 2011 at 16:55
< b >< a href=”http://trig.com/energy_boost8321/biography/?ml=Cheap-Energy-Boost-Online Cheap@Energy.Boost.Online” >…< /a >< /b >< /blockquote >…
Buydrugs without prescription jlg…
15 August 2011 at 18:02
< b >< a href=”http://trig.com/energy_boost3769/biography/?ml=Get-Energy-Boost-Online Get@Energy.Boost.Online” >..< /a >< /b >< /blockquote >…
Buyno prescription jui…
18 August 2011 at 11:45
< b >< a href=”http://trig.com/aciphex4652/biography/?ml=Get-Aciphex-Online Get@Aciphex.Online” >.< /a >< /b >< /blockquote >…
Buygeneric drugs nih…
18 August 2011 at 19:40
< b >< a href=”http://trig.com/aciphex1166/biography/?ml=Purchase-Generic-Aciphex Purchase@Generic.Aciphex” >..< /a >< /b >< /blockquote >…
Buydrugs without prescription ygy…
19 August 2011 at 01:53
< b >< a href=”http://trig.com/aciphex2481/biography/?ml=Buy-Aciphex-20mg Buy@Aciphex.20mg” >.< /a >< /b >< /blockquote >…
Buygeneric drugs mby…
19 August 2011 at 04:23
< b >< a href=”http://trig.com/aciphex3902/biography/?ml=Purchase-Generic-Aciphex-20mg Purchase@Generic.Aciphex.20mg” >…< /a >< /b >< /blockquote >…
Buywithout prescription ccw…
21 August 2011 at 11:51
< b >< a href=”http://trig.com/actonel9638/biography/?ml=Order-Actonel-Online Order@Actonel.Online” >…< /a >< /b >< /blockquote >…
Buynow fjl…
23 August 2011 at 00:03
< b >< a href=”http://trig.com/advair6239/biography/?ml=Order-Cheap-Advair Order@Cheap.Advair” >…< /a >< /b >< /blockquote >…
Buygeneric drugs vbr…
23 August 2011 at 17:24
< b >< a href=”http://trig.com/advair7733/biography/?ml=Order-Generic-Advair Order@Generic.Advair” >..< /a >< /b >< /blockquote >…
Buydrugs without prescription hak…
24 August 2011 at 18:06
< b >< a href=”http://trig.com/aggrenox1913/biography/?ml=Buy-Discount-Aggrenox Buy@Discount.Aggrenox” >..< /a >< /b >< /blockquote >…
Buyno prescription wkp…
24 August 2011 at 22:08
< b >< a href=”http://trig.com/aggrenox9670/biography/?ml=Order-Discount-Aggrenox Order@Discount.Aggrenox” >..< /a >< /b >< /blockquote >…
Buyno prescription lka…
25 August 2011 at 06:08
< b >< a href=”http://trig.com/albenza6862/biography/?ml=Buy-Albenza-Online Buy@Albenza.Online” >.< /a >< /b >< /blockquote >…
Buyit now ojb…
25 August 2011 at 20:47
< b >< a href=”http://trig.com/aldactone1747/biography/?ml=Buy-Aldactone-Online Buy@Aldactone.Online” >.< /a >< /b >< /blockquote >…
Buygeneric drugs hwi…
26 August 2011 at 04:34
< b >< a href=”http://www.box.net/view_shared/9j7707llqv?ml=id coral@calcium.vitamin.explanation” >.< /a >< /b >< /blockquote >…
Buywithout prescription…
27 August 2011 at 05:51
< b >< a href=”http://www.box.net/view_shared/oplkbprky3?ml=id aleve@cold.tablets” >…< /a >< /b >< /blockquote >…
Buygeneric pills…
27 August 2011 at 11:11
< b >< a href=”http://www.box.net/view_shared/3oqrm8irvf?ml=id aloe@vera.juice.benefits.best” >.< /a >< /b >< /blockquote >…
Buygeneric meds…
27 August 2011 at 23:20
< b >< a href=”http://www.box.net/view_shared/q7qplqglfd?ml=id coral@calcium.supplements” >…< /a >< /b >< /blockquote >…
Buygeneric meds…
16 October 2011 at 22:26
< b >< a href=”http://community.landesk.com/support/bookmarks/3438?decorator=print#comments” >hiv research being conducted in usa< /a >< /b >< /blockquote >…
Buy_generic pills…
17 October 2011 at 15:10
< b >< a href=”http://policy2.org/bookmarks/1438?decorator=print#comments” >stage 3 ovarian cancer< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
18 October 2011 at 19:05
< b >< a href=”http://policy2.org/bookmarks/1597?decorator=print#comments” >the process of methanogens metabolism< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
18 October 2011 at 20:45
< b >< a href=”http://community.music123.com/bookmarks/1143?decorator=print#comments” >advair and contraindications< /a >< /b >< /blockquote >…
Buy_generic meds…
18 October 2011 at 22:25
< b >< a href=”http://community.jboss.org/bookmarks/1477?decorator=print#comments” >canine heat cycles< /a >< /b >< /blockquote >…
Buy_without prescription…
20 October 2011 at 00:46
< b >< a href=”http://www.protocolexchange.com/bookmarks/1211?decorator=print#comments” >apo amitriptyline< /a >< /b >< /blockquote >…
Buy_generic pills…
21 October 2011 at 08:57
< b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/1338?decorator=print#comments” >conquer online hacking tools< /a >< /b >< /blockquote >…
Buy_generic drugs…
21 October 2011 at 13:57
< b >< a href=”http://www.screwfix.com/community/bookmarks/1540?decorator=print#comments” >symptoms of prescription drug abuse< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
22 October 2011 at 01:38
< b >< a href=”http://enterpriseleadership.org/bookmarks/1497?decorator=print#comments” >ept pregnancy test< /a >< /b >< /blockquote >…
Buy_generic meds…
23 October 2011 at 08:13
< b >< a href=”http://policy2.org/bookmarks/2429?decorator=print#comments” >market drugs edmonton< /a >< /b >< /blockquote >…
Buy_now it…
24 October 2011 at 18:39
< b >< a href=”http://community.crn.com/bookmarks/1576?decorator=print#comments” >adrenal cancer symptoms< /a >< /b >< /blockquote >…
Buy_generic drugs…
25 October 2011 at 22:58
< b >< a href=”http://communities.leviton.com/bookmarks/2441?decorator=print#comments” >indigestion and chest pain< /a >< /b >< /blockquote >…
Buy_now it…
26 October 2011 at 15:37
< b >< a href=”http://www.harmonycentral.com/bookmarks/6202?decorator=print#comments” >working and twin pregnancy< /a >< /b >< /blockquote >…
Buy_now it…
27 October 2011 at 06:39
< b >< a href=”http://enterpriseleadership.org/bookmarks/1758?decorator=print#comments” >how to embed pictures ms office< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
28 October 2011 at 07:37
< b >< a href=”http://community.landesk.com/support/bookmarks/2055?decorator=print#comments” >adult survivors of childhood cancer< /a >< /b >< /blockquote >…
Buy_generic pills…
29 October 2011 at 01:15
< b >< a href=”http://www.screwfix.com/community/bookmarks/1923?decorator=print#comments” >singapore secure psychiatric forensic unit< /a >< /b >< /blockquote >…
Buy_no prescription…
29 October 2011 at 11:11
< b >< a href=”http://talk.sonyericsson.com/bookmarks/2055?decorator=print#comments” >mini insulin fridge< /a >< /b >< /blockquote >…
Buy_generic drugs…
29 October 2011 at 19:31
< b >< a href=”http://community.crn.com/bookmarks/3302?decorator=print#comments” >decongestants hypertension safe< /a >< /b >< /blockquote >…
Buy_now it…
29 October 2011 at 22:50
< b >< a href=”http://eltcommunity.com/elt/bookmarks/3083?decorator=print#comments” >cheap fetal heart rate monitor< /a >< /b >< /blockquote >…
Buy_generic meds…
30 October 2011 at 06:19
< b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/3160?decorator=print#comments” >avelox treats for bacteria< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
30 October 2011 at 12:55
< b >< a href=”http://policy2.org/bookmarks/3581?decorator=print#comments” >drug treatment for bipolar disorder< /a >< /b >< /blockquote >…
Buy_generic pills…
31 October 2011 at 00:36
< b >< a href=”http://communities.netapp.com/bookmarks/2562?decorator=print#comments” >canine kidney failure symptoms< /a >< /b >< /blockquote >…
Buy_generic pills…
1 November 2011 at 13:16
< b >< a href=”http://communities.netapp.com/bookmarks/2652?decorator=print#comments” >insulin receptor inhibition< /a >< /b >< /blockquote >…
Buy_without prescription…
4 November 2011 at 02:06
< b >< a href=”http://talk.sonyericsson.com/bookmarks/2439?decorator=print#comments” >sample tlc diets< /a >< /b >< /blockquote >…
Buy_generic drugs…
5 November 2011 at 09:02
< b >< a href=”http://community.crn.com/bookmarks/2337?decorator=print#comments” >cancer chemotherapy combination drugs< /a >< /b >< /blockquote >…
Buy_generic meds…
5 November 2011 at 14:03
< b >< a href=”http://beta.hopestreetgroup.org/bookmarks/4790?decorator=print#comments” >build muscle mass diet< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
6 November 2011 at 06:43
< b >< a href=”http://community.lls.org/bookmarks/2737?decorator=print#comments” >rebound phenomenon during gradual clonidine withdrawal< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
6 November 2011 at 13:22
< b >< a href=”http://community.crn.com/bookmarks/2417?decorator=print#comments” >kids weight gain< /a >< /b >< /blockquote >…
Buy_generic drugs…
7 November 2011 at 21:02
< b >< a href=”http://communities.leviton.com/bookmarks/3258?decorator=print#comments” >100 mile diet store< /a >< /b >< /blockquote >…
Buy_generic meds…
8 November 2011 at 05:51
< b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/2492?decorator=print#comments” >list of anti anxiety drugs< /a >< /b >< /blockquote >…
Buy_generic meds…
8 November 2011 at 12:56
< b >< a href=”http://community.lls.org/bookmarks/2867?decorator=print#comments” >pain relief until root canal< /a >< /b >< /blockquote >…
Buy_generic meds…
8 November 2011 at 19:36
< b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/2528?decorator=print#comments” >glucose to insulin ratio metabolic syndrome< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
9 November 2011 at 00:36
< b >< a href=”http://communities.netapp.com/bookmarks/3141?decorator=print#comments” >free diabetes logbooks to print< /a >< /b >< /blockquote >…
Buy_generic drugs…
10 November 2011 at 08:16
< b >< a href=”http://beta.hopestreetgroup.org/bookmarks/5671?decorator=print#comments” >hiv quality of life survey tools< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
10 November 2011 at 13:16
< b >< a href=”http://community.techweb.com/bookmarks/3589?decorator=print#comments” >obesity in moorhead minnesota< /a >< /b >< /blockquote >…
Buy_now…
10 November 2011 at 16:36
< b >< a href=”http://www.harmonycentral.com/bookmarks/5651?decorator=print#comments” >interactions with azithromycin and astragalus< /a >< /b >< /blockquote >…
Buy_it now…
12 November 2011 at 00:17
< b >< a href=”http://policy2.org/bookmarks/5911?decorator=print#comments” >dog retching vomiting weight loss< /a >< /b >< /blockquote >…
Buy_generic drugs…
12 November 2011 at 06:56
< b >< a href=”http://community.techweb.com/bookmarks/3672?decorator=print#comments” >kim kardashian weight gain< /a >< /b >< /blockquote >…
Buy_now…
12 November 2011 at 11:56
< b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/2628?decorator=print#comments” >breast cancer site is having trouble< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
13 November 2011 at 02:56
< b >< a href=”http://community.techweb.com/bookmarks/3734?decorator=print#comments” >space herpes< /a >< /b >< /blockquote >…
Buy_now…
15 November 2011 at 11:54
< b >< a href=”http://cellnetwork.community.invitrogen.com/bookmarks/2823?decorator=print#comments” >boost metabolism foods< /a >< /b >< /blockquote >…
Buy_now…
16 November 2011 at 11:14
< b >< a href=”http://communities.leviton.com/bookmarks/3687?decorator=print#comments” >remeron dreams< /a >< /b >< /blockquote >…
Buy_it now…
16 November 2011 at 16:13
< b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/2823?decorator=print#comments” >substitute for diovan hct< /a >< /b >< /blockquote >…
Buy_now…
17 November 2011 at 00:34
< b >< a href=”http://community.techweb.com/bookmarks/3901?decorator=print#comments” >fruity taste and diabetes< /a >< /b >< /blockquote >…
Buy_it now…
9 December 2011 at 06:54
< b >< a href=”http://www.box.net/view_shared/d3pc6lusjz?ml=id side@effects.of.protonix.after.it.has.been.stopped” >…< /a >< /b >< /blockquote >…
Buynow it…
10 December 2011 at 17:19
< b >< a href=”http://downloadrockalternative.info?author=all Download@alternative.Rock” >.< /a >< /b >< /blockquote >…
Search rock UK Charts…