 |
 |
Sorry if this is a long response, but this is an important issue to me. In my humble opinion, underpowered studies are the most serious problem facing medical research today. Victor Montori writes: >I believe there are small trials being conducted (because of lack of >funds or because the disease is infrequent or because the presumed >effect size is very small) with the purpose of being pooled later. >Pooling is therefore part of the design. I am under the impression >that a European prostate cancer screening trial is being conducted >that way (which in my mind is like a large multicenter RCT). Conducting a bunch of small studies with the intention of eventually pooling them into a meta-analysis is an inefficient practice. Unless each study looks at the same outcome measure, has the same inclusion criteria, and uses the same protocol, then there will problems with heterogeneity in the meta-analysis. And if you take the time to standardize all these details, what you have is a single multi-center trial. I do not know about the European prostate cancer screening trial, but if it improves on meta-analysis by encouraging greater standardization, that's great news. Peter Griffiths writes: >There are, as you point out, many reasons why large samples might not >be available. The opportunity to prospectively plan data pooling on a >national / international scale is just not always there, although I >agree that it should be aspired to. > >What we are left with is the question as to whether or not is complete >ignorance better than some reduction in uncertainty from a small trial >(even if the reduction is marginal), whether it is ethically justified >to impose on patients for small knowledge gains and just how else to >assess effectiveness when sample sizes must be small. There is one solution, though it is unlikely to be adopted. When sample sizes are limited by the nature of the disease (childhood cancer is an example), then we should consider using a larger alpha level. Think in terms of new drugs. A Type I error is allowing an ineffective drug onto the marker and a Type II error is keeping an effective drug off the market. If the current statistical standard effectively prevents most effective drugs from getting to the market (high Type II error rate), then make it easier for ALL drugs to get onto the market by increasing the Type I error rate. I saw a presentation where someone ran a simulation model with certain assumptions about a limited study population and a certain rate of innovation. The simulation showed that under certain circumstances, an alpha level of 50% (!) was best. This rule allowed a lot of ineffective drugs onto the market, but these drugs were rapidly displaced by newer and better drugs. At 25% and 5% alpha levels, fewer ineffective drugs were allowed onto the market, but it also took longer for new innovations to be adopted. Perhaps the current standard of 5% alpha levels is stifling innovation. I don't see journals accepting 50% Type I error levels anytime soon, certainly not on the basis of a single simulation. But it is unclear why we disallow this practice while allowing Type II error levels of 60% or higher. Steve Simon, [log in to unmask], Standard Disclaimer. STATS - Steve's Attempt to Teach Statistics: http://www.cmh.edu/stats %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%