 |
 |
On Aug 3, 2016, at 3:00 AM, Anderson M. Winkler <[log in to unmask]> wrote:Hi David,Ok, I edited the code. Same link: https://dl.dropboxusercontent.com/u/2785709/outbox/palm/palm-alpha98a.tar.gzIt will now take Cohen's d as input (same format as before) and will use the variance internally to convert it to an absolute effect size, which then is subtracted from the COPE before doing the null hypothesis.Note you are the first to test it: I haven't tested at all. Use diff/kdiff3/kompare or similar to compare the code with the non-modified version of PALM.Again: these changes are a one-off and in principle won't be integrated into PALM.Hope it helps.All the best,AndersonOn 3 August 2016 at 00:47, David V. Smith <[log in to unmask]> wrote:Hi Anderson,Thanks -- those are good points and I agree with added clarity/interpretability of the "raw" effect size. I think I'm just getting stuck on the units of the fMRI data, or specifically, the cope images being fed into PALM. Ideally the cope images can be scaled to % signal change measures, but I thought there were a few caveats with that approach (e.g., density of design, peak-to-peak heights)? Even then, % signal change measures don't seem to make sense for reporting connectivity effect sizes.Also, won't the units and scaling of the betas vary across space?Sorry, I think I'm missing something here!Thanks for your help!Best,DavidOn Aug 2, 2016, at 2:55 AM, Anderson M. Winkler <[log in to unmask]> wrote:Hi David,This is something that would have to be changed in the code as it would need to use the variance of the residuals, which isn't known outside the regression. A possible way is to use the option "-saveglm" to have the variance of the contrast (varcope) saved. The varcope is sigma^2/(C'*pinv(M'*M)*C), where C is the contrast and M is the design matrix. You need to know sigma, because Cohen's d is C*beta/sigma, and the -effectdelta requires C*beta. So, use -saveglm in a test with a tiny number of permutations (say, 3, just to have that result), solve the equation above to find sigma, then multiply it by d to find the value that goes in the -effectdelta. This still isn't ideal because the sigma varies across space, and you can already see the consequence: the slopes would vary across space to accommodate a fixed d. To get around it, need to edit the code so as to allow these deltas to operate voxelwise, with a performance penalty.I'd say that, in a way, using the actual, original units is superior to standardised effect sizes in that the original units are immediately interpretable and don't depend on the residuals, which are affected not only by intrinsic variability, but also by measurement error. So it wouldn't be a bad idea to use these as inputs anyway.To put simply: Say you are studying the association between life expectancy and income. It's possible to report something as a Cohen's d = 0.7, or something as that for every $1000 increase on income, there is an associated life expectancy increase by 5 months. Or that every litre of alcohol per capita per year is associated with a 1% increase in the occurrence of cirrhosis, instead of Cohen's of, say, 0.6 (I'm making up these numbers, it's just to show that effect sizes aren't that useful, and the actual slope can often tell more useful information).Cheers,AndersonOn 1 August 2016 at 15:50, David V. Smith <[log in to unmask]> wrote:Thanks, Anderson! One quick followup question: would there be a way to convert/normalize the input data so that the betas are in units of Cohen's d (or some standardized measure)? If not, I'll have to think more about how to translate between the standardized effect size and the actual effect size (in original units).Cheers,DavidOn Jul 31, 2016, at 3:29 AM, Anderson M. Winkler <[log in to unmask]> wrote:Hi David,Please see a custom version of PALM with this feature at the link:It has the option "-effectdelta <filename>", which allows loading a file containing the "d" for testing hypotheses that C'*beta > d, instead of the usual C'*beta > 0.The input file for this option is a Matlab .mat file containing a variable named "effectdelta". This variable should have 3 dimensions: as many rows as inputs (same as the number of -i), as many columns as designs (same as the number of -d) and as many levels as the largest number of t contrasts for any design (i.e., largest number of contrasts across the -t entered). To ensure Octave compatibility, when creating this file, use the option '-v6' in either Matlab or Octave. Note that "d" isn't a standardised effect size: it is the actual effect size, in the original units.As implemented, it works for t and v statistics, but not for F, G, r, or R^2. It doesn't work either for MAN(C)OVA statistics, but it should work for NPC. It should also work with the tail approximation.I am reluctant in making this change definitive as it adds some extra computation at every permutation that isn't otherwise needed. For now it isn't available in the version in the website or Github.The complete description of this approach was presented in a poster during the OHBM2013 (Seattle) by Torben Lund and Kim Mouridsen (Aarhus University, Denmark). I can't find the poster online but you can probably ask Torben via email.Hope it helps.All the best,AndersonOn 8 July 2016 at 14:38, David V. Smith <[log in to unmask]> wrote:Thanks, Anderson! I don't mind waiting.Cheers,DavidOn Jul 8, 2016, at 2:35 AM, Anderson M. Winkler <[log in to unmask]> wrote:Hi David,Ok. If you don't mind waiting, I'll try to have this included at some point by the end of the month. Will update here in the thread then.All the best,AndersonOn 7 July 2016 at 18:29, David V. Smith <[log in to unmask]> wrote:Hi Anderson,Thanks a lot -- that would be great! No rush at all, and I appreciate you taking the time to add it as a feature for PALM.Best wishes,DavidOn Jul 7, 2016, at 2:57 AM, Anderson M. Winkler <[log in to unmask]> wrote:Hi David,Not that I'm aware of. If it's something you really need, it can be added to PALM quite straightforwardly, such that instead of testing if the null is zero, one would test if it's smaller than a certain predefined threshold. If it's something that can be really useful for your analysis, let me know and it can be added (please give me a few weeks, won't have a chance of looking into it now).All the best,AndersonOn 6 July 2016 at 15:38, David V. Smith <[log in to unmask]> wrote:Hi Anderson,Thanks for the feedback. All of those points make sense. I was thinking about this in the context of the non-ironic appendices of Friston's 2012 Neuroimage commentary. If we go with that and assume that d = 0.125 is "trivial", would there be a straightforward way to conduct a protected inference with any of the FSL tools (e.g., randomise, flameo, palm)?Cheers,DavidOn Jul 3, 2016, at 4:26 AM, Anderson M. Winkler <[log in to unmask]> wrote:Hi David,Here are some thoughts:- A half-sample doesn't have half-power. How much power is lost depends on the effect size and on the size of the full sample.- A conjunction of two half-samples is less powerful than the full sample.- Two half-samples aren't completely independent if the strategy used to recruit subjects was the same for both halves and subjects come from the same population.- Meta-analyses generally have a joint null hypothesis that an aggregate effect across the separate (partial) tests is zero. This isn't the same as searching for replication across samples -- the latter is a conjunction.- There are multiple ways in which a sample could be divided. Which to take? Repeating many times and averaging would seem an option, but that also corresponds to a type of meta-analyis.My suggestion is to use the full sample. Trivial effects are indeed a concern, and to get around it one must know what a trivial effect would be, which implies knowing something about the effect size, or about the relevance of that effect. The p-value is only one piece of a larger puzzle.Cheers,AndersonOn 28 June 2016 at 14:34, David V. Smith <[log in to unmask]> wrote:Hello,
I have a stats/fMRI question that isn't specific to FSL. Let's say I'm unsure about the population effect size for the contrast I'm examining, but I want to avoid reporting effects that are trivially small. If I download 200 subjects worth of data, would it be better to a) focus on results derived from the full sample of 200 subjects; or b) split the sample of 200 into two or more groups and focus on results that replicate across subsamples? Or, is there another approach to consider here?
Thanks!
David