Comments on: Population drift http://blog.johndcook.com/2008/02/01/population-drift/ The blog of John D. Cook Sat, 11 Feb 2012 16:19:06 +0000 http://wordpress.org/?v=2.2.1 By: Tubin http://blog.johndcook.com/2008/02/01/population-drift/#comment-30 Tubin Thu, 07 Feb 2008 03:31:05 +0000 http://blog.johndcook.com/2008/02/01/population-drift/#comment-30 My mind immediately went to the challenge: How do you adjust the ongoing analysis in an adaptive trial to monitor for the presence of population drift? You lose some power by adding another variable, but it might be worth it. In the end, though, I (as a clinician) am acutely aware that we are discussing subtleties when the vast majority of daily clinical actions are taken on the basis of virtually no evidence whatsoever. Either the doc is not aware of the evidence, or the research simply hasn't been done. The possibility (probability?) that population drift undermines the believability of what evidence we have - well, my mind shuts down and refuses to contemplate the implications seriously. Should we spend research dollars trying to pursue new hypotheses, or continuously retest the ones we've done before? Come to think of it, this might be an argument for the trend of drug comparison trials rather than placebo trials; at least that way we have ongoing evaluation of old therapies as we test new ones. My mind immediately went to the challenge: How do you adjust the ongoing analysis in an adaptive trial to monitor for the presence of population drift? You lose some power by adding another variable, but it might be worth it.

In the end, though, I (as a clinician) am acutely aware that we are discussing subtleties when the vast majority of daily clinical actions are taken on the basis of virtually no evidence whatsoever. Either the doc is not aware of the evidence, or the research simply hasn’t been done. The possibility (probability?) that population drift undermines the believability of what evidence we have - well, my mind shuts down and refuses to contemplate the implications seriously.

Should we spend research dollars trying to pursue new hypotheses, or continuously retest the ones we’ve done before?

Come to think of it, this might be an argument for the trend of drug comparison trials rather than placebo trials; at least that way we have ongoing evaluation of old therapies as we test new ones.

]]>