Just throwing out a couple of extra thoughts for discussion:

I think perhaps it would be useful to make a distinction between
information that is necessary for other researchers to be able to
replicate an experiment, and information not necessarily needed for
replication, but that a reviewer might want to see. The former would
include all the details of the experimental design and processing steps
that have been mentioned in this discussion, and could be made manditory
(at least as supplemental info.).

The latter, on the other hand, is a rather open-ended category, and it
might be best left to reviewers and editors to request such information
that they feel would enable them to properly assess the quality of the
submitted paper. It is unavoidable that there needs to be a large
element of trust involved in the review process. i.e. if researchers
claim that they have performed the appropriate checks when using a
specific analytic procedure, then unless a reviewer has very good
grounds for doubting those claims, I don't think it would be
appropriate, or practical, to demand "proof". Perhaps some guidelines
could be provided as to what information (e.g SNR, error distributions
etc) would be useful for most papers, without making the provision of
such information manditory.

Tom

-------------------------------------------------
Tom Johnstone
Waisman Laboratory for Brain Imaging and Behavior
Waisman Center
University of Wisconsin-Madison
Tel. +1 608 263 2743 Fax. +1 608 265 8737
[log in to unmask]

----- Original Message -----
From: Torben Ellegaard Lund <[log in to unmask]>
Date: Wednesday, February 16, 2005 1:07 pm
Subject: Re: [SPM] Any Papers on Presenting fMRI Results?

> Hi Alexa and SPM'ers
> 
> 
> On 16 Feb 2005, at 15:37, Alexa Morcom wrote:
> >
> 
> >   
> > Do we need to give more detail than, for example, papers based on 
> > behavioural or ERP data? It's not usually considered necessary to 
> > include residual plots in these cases.
> >
> 
> I don't know much about ERP data. But I do feel that before a 
> consensus 
> is reached, with regard to the hundreds of possible choices made 
> before 
> the beautiful colour-overlayed image pops up, it is mandatory to 
> know 
> as much as possible about relevant processing information. And as 
> we 
> don't know yet which factors are the most relevant ones we need to 
> include a lot of information. If your residuals are i.i.d. fine. 
> But if 
> they are not your results are biased and the reader should know that!
> 
> >   
> > Do we want to make imaging papers intelligible to non-expert-
> imagers 
> > and to non-imagers?
> 
> I would definitely not want to hide crucial information, and risk 
> NeuroImage to loose its impact, just because some non-expert-imager 
> cannot understand every single word.
> 
> 
> >  I think this is important, and so would want to avoid too much 
> > technical detail in papers - such as design matrices, which in 
> any 
> > case are software-specific - just enough to enable replication 
> and to 
> > make it clear that the important issues have been dealt with 
> > appropriately.
> >   
> 
> But what is appropriate when there are few issues where consensus 
> has 
> yet been achieved?
> 
> 
> > I don't know about supplementary web-based information - it's 
> nice, 
> > but I imagine that not everyone has easy access to it
> 
> as mentioned earlier it can normally it is provided by the Journal
> 
> 
> > - and one can always contact the authors.
> 
> that is not really a good alternative for e.g. meta-analysis
> 
> 
> >  If it's really important, shouldn't it be in the paper?
> >
> 
> You could argue so, but have you seen how thick recent issues of 
> NeuroImage is?
> 
> 
> 
> Hope this was not too emotional...
> 
> Torben
> 
> 
> 
> >  
> > If anyone's interested, something like this was done for ERPs by 
> > Picton et al (2000, Psychophysiology. 2000 Mar;37(2):127-52)
> >  
> > Alexa
> > -----Original Message-----
> > From: SPM (Statistical Parametric Mapping) 
> > [mailto:[log in to unmask]]On Behalf Of Torben Ellegaard Lund
> > Sent: 16 February 2005 01:17
> > To: [log in to unmask]
> > Subject: Re: [SPM] Any Papers on Presenting fMRI Results?
> >
> > Interesting discussion this one
> >
> > I guess Tom will be too polite to mention this. But I think it 
> would 
> > be very convincing if statistic maps were accompanied by results 
> of an 
> > SPMd analysis to demonstrate normal and white residuals (or lack 
> > thereof!). Unfortunately for first level analysis it is really 
> only an 
> > option for the SPM folks. But I think SPMd would serve as a great 
> > quality check.
> >
> >  best
> > torben
> >
> >
> > Torben E. Lund
> > Danish Research Centre for MR
> > Copenhagen University Hospital
> > Kettegaard Allé 30
> > 2650 Hvidovre
> > Denmark
> > email: [log in to unmask]
> > webpage: http://www.drcmr.dk
> >
> > On 16 Feb 2005, at 01:35, Matthew Brett wrote:
> >
> >
> > Hi,
> >
> > I would put in a plea for continuous activation maps to be made
> > available - and displayed in the paper or supplementary material. 
> The> thresholded maps we are all used to can be seriously misleading:
> >
> > Jernigan TL, Gamst AC, Fennema-Notestine C, Ostergaard AL. More
> > "mapping" in brain mapping: statistical comparison of effects. Hum
> > Brain Mapp. 2003 Jun;19(2):90-5
> >
> > In my experience, continuous maps also give a much clearer 
> picture of
> > the quality of the data.
> >
> > Also, it seems to me that any ROIs used should be made available 
> > online.
> >
> > Best,
> >
> > Matthew
> >
> > On Mon, 14 Feb 2005 17:36:23 -0500, Thomas E Nichols 
> > <[log in to unmask]> wrote:
> >
> > Max,
> >
> >
> > Is anyone aware of papers about presenting results for fMRI studies?
> > Specifically I'm looking for any attempts that have been made to
> > standardize what is reported and how.
> >
> > I don't know of any such efforts, but I think it's badly needed. I
> > was once asked by an editor for such standards and started to 
> make a
> > list of statistical and non-statistical issues. I'd love to hear
> > comments on such guidlines.
> >
> > -Tom
> >
> > -- Thomas Nichols -------------------- Department of Biostatistics
> > http://www.sph.umich.edu/~nichols University of Michigan
> > [log in to unmask] 1420 Washington Heights
> > -------------------------------------- Ann Arbor, MI 48109-2029
> >
> > All papers should give sufficient detail so that if the reader were
> > armed with the authors' data they could reproduce the results. Some
> > important items:
> >
> > 1. What voxel-wise statistic image threshold was used? Corrected or
> > uncorrected? FWE or FDR?
> >
> > 2. Was cluster size inference used? If so, what is the
> > cluster-defining statistic image threshold? What is the cluster
> > size threshold (in voxels) and significance (corrected or
> > uncorrected).
> >
> > 3. How many voxels corrected for? Whole brain voxel count, or
> > sub-volume count for 'Small Volume Correction'. If small volume
> > correction, define how the sub-region was defined.
> >
> > 4. If random field theory is used, what is the smoothness (FWHM,
> > x,y,z)? What is the RESEL count? (This allows one to independly
> > recompute the corrected threshold)
> >
> > Not directly related to the statistics, but crucial for any complete
> > reporting are:
> >
> > a. Basic image properties: image dimensions and voxel size.
> > Properities of data as acquired *and* after intersubject
> > registration (aka Spatial Normalization). For PET/SPECT, image
> > reconstruction smoothness parameter (e.g. 'ramp filtered', 'Hanning
> > filter, *** mm cutoff').
> >
> > b. Was slice timeing correction used?
> >
> > c. Smoothing applied. At 1st level and 2nd level if done twice.
> >
> > d. Basic intrasubject registration info. What software, what sort of
> > interpolation.
> >
> > e. Basic itersubject registration parmaeters. Affine/Linear? If so,
> > how many parameters (9 or 12, typically). If Nonlinear, 'how'
> > nonlinear? (E.g. with AIR, you specify a polynomial order; with
> > SPM, you specify a basis size, like 3x2x3). Regularization
> > setting. What interpolation?
> >
> > This may sound like a lot, but they are all very basic parameters 
> and> can be concisely reported. They also can be reported in detail 
> in one
> > publication from a lab and then cite that publication for details 
> that> haven't changed.
> >
>