Are you saying that given time if there is a bias it will show in other ways and that a premature assumption also has bias potential and so we should focus on areas that need work and are more amenable to reconstruction?
Amy
Amy Price PhD
Empower 2 Go
Building Brain Potential
Http://empower2go.com
Sent from my iPad
On 9 Nov 2012, at 02:30 PM, "Steve Simon, P.Mean Consulting" <[log in to unmask]> wrote:
> On 11/9/2012 12:39 PM, Ahmed Abou-Setta, M.D. wrote:
>
>> The problem is often in the literature we often see that some areas
>> are dominated by certain groups of researchers, and in certain
>> cases, the majority of the evidence come from these isolated pockets
>> of researchers and by sheer numbers the evidence becomes in favor of
>> whatever they state.
>
> So let's make the example more concrete. Harvard does a lot of research, so they might qualify as an "isolated pocket." You want to downgrade research just because it comes from Harvard? The risk of biasing your results because you are cherry picking far outweighs the risk that Harvard has a systematic and predictable bias in research.
>
> If one group of researchers do a lot of research in an area, we should be grateful for their contributions and not suspicious. Having too much data from a single source is not ideal, of course, but far too often we suffer not from that but rather from having too little data from any source.
>
> A systematic deviation between Harvard and everyone else could mean a lot of things. It could represent a geographic effect, it could represent the better quality of work done at Harvard, it could represent a methodological approach to research that is unique to Harvard, or it could be bias. I, for one, would hate to make an accusation of bias without first investigating all the other options thoroughly.
>
> Let the meta-analysts report various subgroups and comment on them as a possible source of heterogeneity. Clearly having a subgroup of Harvard research that differs from everyone else will limit generalizability of the findings, with or without an accusation that Harvard is biased.
>
> Furthermore, "isolated pockets" is so ill-defined as to lead to potential spurious findings. Do you look at a "Harvard" effect or an "Ivy League" effect or an "East Coast" effect or a "private school" effect? Even if you hypothesize a "Harvard" effect, how do you define it? Is it research conducted by researchers currently working at Harvard, or should you include researchers who recently graduated from Harvard as well? If so, how recent? How about all the people who tried and failed to get tenure at Harvard and have moved elsewhere? Do you look at the affiliation of the first author, or examine affiliation from any of the authors? It sure sounds like the worst type of subgroup analysis to me. Financial ties, though not completely unambiguous, are still far better defined, by comparison.
>
> So I'd encourage you to drop your effort to show that isolated pockets of researchers are biasing our meta-analyses.
>
> Steve Simon, [log in to unmask], Standard Disclaimer.
> Sign up for the Monthly Mean, the newsletter that
> dares to call itself average at www.pmean.com/news
>
|