I have encountered this in familiar fields for me and have wondered how to deal with this. It concerns me that in unfamiliar fields I would not know the difference and that my perspective could be slanted to error. I am glad you are looking at this as this is a most valid question
Best
Amy
Amy Price PhD
Empower 2 Go
Building Brain Potential
Http://empower2go.com
Sent from my iPad
On 9 Nov 2012, at 01:39 PM, "Ahmed Abou-Setta, M.D." <[log in to unmask]> wrote:
> Let me elaborate the problem further.
>
> Theoretical case report:
>
> A series of 50 trials are conducted to test the effect of Drug A versus B on
> all-cause mortality. There is negligible statistical heterogeneity between
> the trials (I2 = 0, P = 0.81) and the results show no significant difference
> between the interventions. Even so, 10 of the trials are by one group, while
> all the other trials are done by independent groups. When segregated, the
> group of 10 trials performed by group X show a significant effect in favor
> of Drug A, while the pooled result from the remaining 40 trials show a
> significant effect in favor of Drug B (Test for subgroup differences: P =
> 0.08, I2 = 68%. Group X has not declared its affiliation, or receiving
> funding or trial support, from any pharmaceutical company or obvious source
> of bias.
>
> So do we believe that Drug A is better than Drug B, both are equivalent or
> Drug B is better than Drug A? Also, can we assume that there may underlying
> confounders (e.g. author bias) that could be affecting Group X's results
> compared with everyone else's?
>
> The problem is often in the literature we often see that some areas are
> dominated by certain groups of researchers, and in certain cases, the
> majority of the evidence come from these isolated pockets of researchers and
> by sheer numbers the evidence becomes in favor of whatever they state.
>
> Any thoughts?
>
> Ahmed
>
>
> -----Original Message-----
> From: Evidence based health (EBH)
> [mailto:[log in to unmask]] On Behalf Of Steve Simon,
> P.Mean Consulting
> Sent: Friday, November 09, 2012 12:08 PM
> To: [log in to unmask]
> Subject: Re: Author Bias and Objective Outcome Measures
>
> On 11/8/2012 2:48 PM, Ahmed Abou-Setta, M.D. wrote:
>
>> I am looking into the issue of 'author bias' especially with the
>> reporting of 'objective outcomes' (e.g. mortality). I know that bias
>> from known sources like sponsorship (e.g. pharmaceutical industry) has
>> been well investigated, but about just purely 'author bias'.
>> Often authors are biased one way or another for a vast number of
>> reasons including but not limited to personal beliefs of
>> efficacy/effectiveness, prior observations in clinical practice, etc.
>> I am looking for publications which measure or test this bias
>> especially for objective outcomes since it's much easier to bias
>> subjective outcomes (e.g. pain scores) than objective ones (e.g.
>> mortality) especially if the trials were randomized.
>
> I believe you have posed an unanswerable question. If authors are "biased
> one way or another," that means that the bias is inconsistent.
> As such, it will be indistinguishable from pure error.
>
> You need to have a consistent direction to assess bias. The industry funding
> studies provide that direction, because authors tend to get support from a
> company selling one of the drugs in the study but not very often do they
> also get support from the company selling the comparison drug.
>
> If you're looking at a mortality study and you find an error, how would you
> decide whether it was an inadvertent error versus an error caused by
> conscious or unconscious biases in the researcher? You'd have to have some
> indication of what that researcher's biases were, and without a mind reader,
> I can't imagine how you'd do it.
>
> You might be able to look at a more narrowly focused question, perhaps.
> For example, Does a study comparing a surgical to a non-surgical outcome
> tend to be biased towards the surgical outcome when the lead author is a
> surgeon? You could also look at biases towards a statistically significant
> effect. But "author bias" defined very generally is impossible to pin down
> because you can't pin down the direction of the bias for any individual
> author.
>
> Steve Simon, [log in to unmask], Standard Disclaimer.
> Sign up for the Monthly Mean, the newsletter that dares to call itself
> average at www.pmean.com/news
>
|