Alle Meije Wink wrote:
> As you can read in Will's e-mail, it is perfectly legal to report
> results based on a p=0.08 threshold, as long as you accurately
> describe what you're doing. Your next big problem then is getting
> your paper past the reviewers...
This thread gives me an excuse to raise an issue I've been meaning to
ask about. FDR and FWER are different standards, and provide
different assurances. I don't really know much about the historical
basis for the gold standard of FWER=0.05, but it's certainly in part
cultural (how the standard is imposed varies across sub-fields even
within a discipline). I wonder if there's been work (in fMRI or
elsewhere) on which standard is "better" in the sense of maximizing
the rate of growth of knowledge (or something like that). I can't
imagine there's a way to do this uncontroversially, but it seems like
a problem that could be modeled in fMRI, given a raft of assumptions
and a decent amount of data for estimating parameters.
One of the reasons I've wondered about this is that basic cognitive
fMRI seems like an area that would benefit from relaxed standards in
reporting. Most studies are under-powered, so it could be imagined
that Type II error is more rampant than Type I (although dubious
methods are also rampant). At the same time, if the major constraint
is the rate at which journals can publish articles, then relaxed
standards aren't going to help all that much -- the total number of
articles will stay the same, but it will just allow some better
decisions around the boundaries.
To be a hair more on-topic, as Alle Meije noted above, there's no
reason you can't use whatever standard appeals to you in reporting.
Significance thresholds are just one piece of the puzzle in evaluating
how scientifically informative a data set is. Reviewers are entitled
to make decisions about what they consider meaningful, and journals
are entitled to set policies as well. I'm personally willing to
entertain whatever assurances a study provides, and then decide if I
feel an article as a whole has sufficient impact in the sense of its
effect on the state of scientific knowledge. But I've never had an
article to review that used FDR, and I don't know what the relevant
journal policies are, so I'll stop now.
dan
|