Hey there Dan,
You've started talking about something very interesting. But isn't a
critical concept that type I and type II errors are frequentist
ideas, and within the frequentist paradigm one has no idea about
the likelihood of type I or type II errors occurring (one only
knows conditional probabilities), and therefore no idea of which
one is more of "a problem"? That is, it sounds like you are
appealing to prior knowledge about the likelihood of the null being
true, and hence stepping outside of frequentist inference. If you'd
like to have results which relate to a degree of confidence that
the null hypothesis is false or true (i.e., a probability that the
null is false/true), I think Bayesian inference is the right
philosophical and mathematical terrain, not type I and type II
error rates. What do you think?
Eric
Quoting Daniel Y Kimberg <[log in to unmask]>:
> Russ Poldrack wrote:
> > Dan - I agree that most fmri studies are probably underpowered,
> but
> > I don't beleive that loosening our Type I error rate is the way
> to
> > fix this problem. Rather, people need to run enough subjects
> that
> > they have power to find a sufficient effect size at a
> reasonable
> > Type I error rate.
>
> Tom agreed, and I do too. In principle at least, I'd rather see
> better-powered studies too (and I have been, it's just slow).
> Taking
> costs into account, the two ways I can think of to do this are:
> (a)
> more funding; and (b) fewer scientists. Neither seems like a
> great
> idea, although it could be argued that a small number of
> high-quality
> studies would be preferable to a large number of bad ones (given
> that
> we could find some other kind of work for all the out-of-work
> scientists).
>
> My earlier question was meant to take the problem of underpowered
> studies as a given. I was just wondering if, given that we can't
> force people to run better powered studies, but we can adjust how
> we
> review articles, maybe Type II error is more of a problem than
> Type I.
> Or would be, if people weren't so clever about finding ways to
> make
> Type I errors.
>
> dan
>
|