Mark,
How are you testing significance of the indirect effect? Is it Sobel or bootstrapping? 

If you switch things around and the model is still significant then you have limited support from your data for your model. The same data supporting a non-plausible model (X->Y->M) decreases support for your theory. If you present the results you mention here I think you need to be very explicit about your finding from the alternate model and discuss why there may be support for X->Y->M. Maybe there are unmeasured mediators or moderators? You may also have the following effect going on:

Y = M + X + MX

This tests whether the relationship between brain and cognition differs between the age groups? Because the simple mediation assumes that the relationship between brain and cognition is the same for both age groups and that age simply moves people along this linear relationship. 

What I like to do is to put the data into SPSS and test different models easily with Andrew Hayes "Process" tool. Where you can test all kinds of different moderated-mediation models.. 


take care,
Jason


On Mon, Apr 22, 2013 at 10:35 AM, Tseng Mark <[log in to unmask]> wrote:
Thanks a lot Jason.

For 2), I did test another model, that is, X-->Y-->M (it is originally X-->M-->Y, where the p value for mediation effect is about 0.01) , and now the p value for mediation effect is 0.04. It's higher that 0.01 but also significant.

Does that mean that I can't (or can) be confident about my original model? But the new model  X-->Y-->M seems doesn't make sense (X=age, Y=a cognitive variable, M = a brain structure) because usually we think age causes structural changes which results in changes in cognition, not the reverse?


2013/4/22 Jason Steffener <[log in to unmask]>
Dear Mark,
 I would suggest reading  papers by Preacher and Hayes about mediation. They have quite a few easy to read discussions about the procedures. 

And to address the reviewers points: 
1) This is up to you. It is only theory that can support a mediation analysis performed with cross-sectional data. Do the relationships that you are testing make sense? Is there theory to support an X-->Y relationship, an X-->M relationship,  an M--Y relationship? and what does it mean that M mediates the relationship between X and Y? This needs to be clearly explained and supported with theory otherwise you have the reviewers correct concern.

2) This should be explored by testing alternate models of the X,M,Y relationships. See Salthouse 2012 "Neuroanatomical substrates of age-related cognitive decline"

3) This is based on the theoretical underpinnings of your model and the support that you data gives for the model.

I hope this helps,
Jason



On Mon, Apr 22, 2013 at 8:53 AM, Tseng Mark <[log in to unmask]> wrote:
Dear Smith, can I know which paper it is? yes we do use the mediation toolbox developed by Tor Wager. Thanks a lot.


2013/4/22 Stephen Smith <[log in to unmask]>
Hi -  there's a nice paper by Tor Wager on that topic that I'd recommend you read?
Cheers.


On 22 Apr 2013, at 06:41, Mark <[log in to unmask]> wrote:

Hi,

This is a followup question about a mediation analysis in our study. One of the reviewers asked for questions below and I wonder if someone could help, because I don't know how to calculate the parameter the reviewer requested. The questions are:

(1) Is the mediation analysis sensible?  
(2) Can you provide an assessment of the credibility of the outcomes of the analysis?  
(3) How confident are they that the result would be replicated in another sample?

How can I calculate the sensibility, credibility, and confidence?

Thanks in advance.

Mark



---------------------------------------------------------------------------
Stephen M. Smith, Professor of Biomedical Engineering
Associate Director,  Oxford University FMRIB Centre

FMRIB, JR Hospital, Headington, Oxford  OX3 9DU, UK
+44 (0) 1865 222726  (fax 222717)
[log in to unmask]    http://www.fmrib.ox.ac.uk/~steve
---------------------------------------------------------------------------