Hi all,
I'm responding late to this active discussion. I've incorporated
everyones' comments as best possible into the web page
http://www.sph.umich.edu/~nichols/NIpub
It sounds like there's definite interest in pursuing this. If you'd like
to meet at OHBM and work on massaging this into a paper, please email me
(off the list).
Response to everyone comments are collected below.
-Tom
PS: My apologies to Jack Kelly for not jumping to Wiki... I started
revising my HTML before I saw your Wiki page. I'll try to move
it there asap.
Torben wrote:
> But I think it would be very convincing if statistic maps were
> accompanied by results of an SPMd analysis to demonstrate normal and
> white residuals (or lack thereof!). Unfortunately for first level
> analysis it is really only an option for the SPM folks. But I think
> SPMd would serve as a great quality check.
I'm all for promoting my own work, but I think that diagnosis is
not a cook-book procedure and it would be hard to have any simple
summary that you could post as supplementary information.
This is where I think just the social forces need to come into play...
the results in a paper that has critically evaluated its modeling
assumptions will carry greater weight than the results form one
that hasn't.
Dara wrote:
> unfortunately, many people reporting methods and results don't
> have an understanding of the underlying statistical operations
> underlying the software interface they use. for example, many SPM
> users may not know whether they are using OLS or OLS with variance
> correction. in the guidelines, it might help to include a simple
> mapping between the buttons they press and the underlying statistical
> functions/descriptions for each major analysis package. i admit that
> this smacks of spoonfeeding, and users should really know what they're
> doing, but for the sake of having informative methods descriptions,
> it might help to have this.
This is a good idea, though it will quickly become dated as new
software comes out. Perhaps in an appendix to a guidelines paper?
Alexa wrote
> Do we want to make imaging papers intelligible to non-expert-imagers
> and to non-imagers? I think this is important, and so would want to
> avoid too much technical detail in papers - such as design matrices,
> which in any case are software-specific - just enough to enable
> replication and to make it clear that the important issues have been
> dealt with appropriately.
I agree, but...
> I don't know about supplementary web-based information - it's nice,
> but I imagine that not everyone has easy access to it - and one can
> always contact the authors. If it's really important, shouldn't it be
> in the paper?
disagree, that supplemental info on the web is relatively painless,
and offers a wealth of information for those interested.
Ian wrote
> I think perhaps it would be useful to make a distinction between
> information that is necessary for other researchers to be able to
> replicate an experiment, and information not necessarily needed for
> replication, but that a reviewer might want to see. The former would
> include all the details of the experimental design and processing steps
> that have been mentioned in this discussion, and could be made manditory
> (at least as supplemental info.).
[...]
> Perhaps some guidelines
> could be provided as to what information (e.g SNR, error distributions
> etc) would be useful for most papers, without making the provision of
> such information manditory.
Yes, I agree, and this is the structure I've tried to put into the
web page. Making the distinction between 'Methods' guidelines, and
'Results' guidelines.
Dan wrote, in short...
> I think it's worth considering, by the way, that the concept of the
> information required for replication is not always clear-cut.
Which I agree with. What ever comes of this will just be recommendations.
It's success will be measured by the degree of consuses that goes
into it.
Jack (aka Daniel Kelly) wrote
> I've taken the liberty of starting a page on the SPM Wiki concerning
> this issue:
>
> http://en.wikibooks.org/wiki/SPM-Information_to_include_in_papers
Excellent! The only comment I have is that I feel strongly that
the guidelines should be software-independent. Both so they have
some longevity past the current softwar versions, and so that non-SPM
users will be drawn to them as well.
But just as a forum to get things rolling, using this SPM-Wiki page
sounds like a good starting place. Now I just need to take the time
to convert my HTML to Wikispeak (ug).
|