Dear Clare,
Thank you for the extra details. I had noticed the mention to NNDb in
your script. This particular setting makes the design not particularly
efficient, especially regarding the temporal high-pass filter. How are
these datasets traditionally analysed?
Let us know if the crashes still occur when running the analysis one
participant at a time in different MATLAB sessions.
If the analysis mask saved by SPM looks OK then hopefully it will be
fine despite your masking threshold setting. SPM still ignores voxels
with constant values over time, which, I guess, is what happens in your
case with preprocessed data automatically masked.
Best regards,
Guillaume.
On 31/08/2021 09:34, Lally, Clare wrote:
> Dear Guillaume,
>
> Thank you for your response! That's right about the data being over 7000 scans. The design is less neat... I am using open-access pre-processed data from the Naturalistic Neuroimaging Database, where participants were scanned whilst they watched a feature-length movie. The events that I am modelling are instances of particular movie characters speaking, so the 'trials' are unequal sizes and distributed differently across the run.
>
> I will try running one participant at a time, and I will try to keep note of when the script crashes.
>
> I remember you mentioned this issue with disabling masking before, I use a mask when I conduct the next stage of my analyses (multivariate) but it sounds like I need to do something at this stage to. My SPM script produces the binary mask.nii and the output is sensible. Can I apply this mask at this stage? Alternatively I could exclude any values that are zero exactly (as all values outside of the brain are zero) although I worry I'd accidentally exclude values inside the brain as well. Which would you advise and please could I have some guidance on how to include this in the code? Thank you!!!
>
> All the best,
> Clare
>
> On 27/08/2021, 10:49, "Flandin, Guillaume" <[log in to unmask]> wrote:
>
> Dear Clare,
>
> A two hour recording in a single session is long. Do I understand it
> correctly that there is on average one trial every 30 seconds and that
> they are spread across 24 conditions? With a TR of 1s, this means that
> you have over 7000 scans?
>
> While it should not happen, I presume that the error is due to out of
> memory when handling covariance matrices. Do you know at which stage do
> the crashes occur? Not a satisfactory answer but it might help if you
> close and restart MATLAB in between the analysis of each subject (there
> is also a 'pack' command in MATLAB but I don't know how well it would
> work in your case).
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mathworks.com%2Fhelp%2Fmatlab%2Fref%2Fpack.html&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7Cb0c72fcb47d741ed81b208d96c5a2c5d%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637659956810915069%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pS52mk5DefEqk%2B%2BxaEQLdrOcRCSQnQdgaNSK%2BwLQKsg%3D&reserved=0
>
> One thing I notice in your script is that you disabled masking:
> jobs{1}.stats{1}.fmri_spec.mthresh = -Inf;
> jobs{1}.stats{1}.fmri_spec.mask = {''};
> From the comment in your code, I understand that the data have already
> been mean-centred during preprocessing hence the change but make sure
> outside of the brain is not considered during the GLM analysis otherwise
> provide an explicit mask.
>
> Best regards,
> Guillaume.
>
>
> On 26/08/2021 11:26, Lally, Clare wrote:
> > Hi everyone,
> >
> >
> >
> > Please could I have some advice on how to make my first level model
> > estimation batch script more computationally efficient? I am working
> > with a large dataset, in which participants have 2 hours of functional
> > data in a single run. In my model, I have 242 events distributed across
> > 24 conditions.
> >
> >
> >
> > I have been running the script below on a 3.6 GHz 16 GB iMac, but it is
> > taking over 8 hours per participant. Also, the script routinely crashes
> > with the generic ‘MATLAB has encountered an error and needs to close’,
> > which means I have to restart the script from the current participant.
> > However, it does run the whole way through otherwise.
> >
> >
> >
> > I suspect the script runs slow and crashes because the code is producing
> > things that don’t need to be created in the first place, or that can be
> > removed from memory once they have served their purpose.
> >
> >
> >
> > I’d be very grateful for any advice on how to check this or what to
> > change. Thank you!
> >
> >
> >
> > Clare
> >
> >
> >
> >
> >
> > %% Univariate matrix items vs. baseline - estimate the model
> >
> >
> >
> > % Specify whether to conduct model estimation (time-consuming)
> >
> > ESTIMATE_GLM = 1; % Set this to zero to check the script without
> > estimating
> >
> >
> >
> > % Scanning/timing details
> >
> > TR = 1;
> >
> > timings = readtable(fullfile(timings_path, 'Contrasts_file.csv'));
> >
> > runs = unique(timings.Run);
> >
> > conditions = unique(timings.ConditionName);
> >
> > runs_n = length(runs);
> >
> > conditions_n = length(conditions);
> >
> > onsets_n = length(timings.Onset);
> >
> >
> >
> > % Start for loop for first level analyses here
> >
> > for p = 1:numel(ppt)
> >
> >
> >
> > % Progress message
> >
> > disp(['Processing ', char(ppt(p)), ' ...'])
> >
> >
> >
> > % Define locations for data and outputs
> >
> > func_path = fullfile(data_path, char(ppt(p)), 'func');
> >
> > funcs = dir(fullfile(data_path, char(ppt(p)), 'func',
> > [char(ppt(p)) '*_bold_no_blur_no_censor.nii']));
> >
> > funcs = funcs.name;
> >
> > numScans = numel(spm_vol(fullfile(func_path,funcs)));
> >
> > anat = fullfile(data_path, char(ppt(p)), 'anat',
> > [char(ppt(p)) '_t1w_mni_alignment.nii']);
> >
> > output_path = fullfile(data_path, char(ppt(p)), output_folder);
> >
> >
> >
> > % Begin creating jobs structure
> >
> > jobs{1}.stats{1}.fmri_spec.dir = cellstr(output_path);
> >
> > jobs{1}.stats{1}.fmri_spec.timing.units = 'secs';
> >
> > jobs{1}.stats{1}.fmri_spec.timing.RT = TR;
> >
> > jobs{1}.stats{1}.fmri_spec.timing.fmri_t = 16;
> >
> > jobs{1}.stats{1}.fmri_spec.timing.fmri_t0 = 1;
> >
> >
> >
> > % Create multiple conditions .mat file for each run
> >
> > for r = 1:size(runs_n, 1)
> >
> > % Create empty structures for onsets and durations
> >
> > names = conditions';
> >
> > onsets = cell(1, size(conditions,1));
> >
> > durations = cell(1, size(conditions,1));
> >
> >
> >
> > % For each condition
> >
> > for c = 1:size(conditions,1)
> >
> >
> >
> > % For each onset of an event
> >
> > for x = 1:onsets_n
> >
> > if isequal(timings.ConditionName(x), conditions(c)) &&
> > timings.Run(x) == r
> >
> > onsets{c} = double([onsets{c} timings.Onset(x)]);
> >
> > durations{c} = double([durations{c}
> > timings.Duration(x)]);
> >
> >
> >
> > end
> >
> > end
> >
> > end
> >
> >
> >
> > % Save the structure as a .mat file for each run per participant
> >
> > save (fullfile(output_path, [mat_prefix, char(ppt(p)), '_run',
> > num2str(runs_n(r))]), ...
> >
> > 'names', 'onsets', 'durations')
> >
> >
> >
> > % Get frames for each run using spm_select - add run-related
> > file indexing to the line below if needed
> >
> > files = spm_select('ExtFPList', func_path, funcs, 1:numScans);
> >
> >
> >
> > % Fill in session information within jobs structure
> >
> > jobs{1}.stats{1}.fmri_spec.sess(r).scans = cellstr(files);
> >
> > jobs{1}.stats{1}.fmri_spec.sess(r).cond = struct('name',
> > {}, 'onset', {}, 'duration', {}, 'tmod', {}, 'pmod', {});
> >
> > jobs{1}.stats{1}.fmri_spec.sess(r).multi =
> > cellstr(fullfile(output_path, [mat_prefix, char(ppt(p)), '_run',
> > num2str(runs_n(r)), '.mat']));
> >
> > jobs{1}.stats{1}.fmri_spec.sess(r).regress = struct('name',
> > {}, 'val', {});
> >
> > jobs{1}.stats{1}.fmri_spec.sess(r).multi_reg = {''};
> >
> > jobs{1}.stats{1}.fmri_spec.sess(r).hpf = 128;
> >
> > end
> >
> >
> >
> > % Fill in the rest of the jobs fields
> >
> > jobs{1}.stats{1}.fmri_spec.fact = struct('name', {}, 'levels', {});
> >
> > jobs{1}.stats{1}.fmri_spec.bases.hrf = struct('derivs', [0 0]);
> >
> > jobs{1}.stats{1}.fmri_spec.volt = 1;
> >
> > jobs{1}.stats{1}.fmri_spec.global = 'None';
> >
> > jobs{1}.stats{1}.fmri_spec.mthresh = -Inf; % For mean-centred
> > functional images - needed for NNDb
> >
> > jobs{1}.stats{1}.fmri_spec.mask = {''};
> >
> > jobs{1}.stats{1}.fmri_spec.cvi = 'AR(1)';
> >
> >
> >
> > % Navigate to output directory, specify and estimate GLM
> >
> > cd(output_path);
> >
> > spm_jobman('run', jobs)
> >
> >
> >
> > if ESTIMATE_GLM == 1
> >
> > load SPM;
> >
> > spm_spm(SPM);
> >
> > end
> >
> > end
> >
> >
> >
> > Clare Lally (Post-doctoral Research Fellow)
> > UCL Vocal Communication Laboratory
> > <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.carolynmcgettigan.com%2F&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7Cb0c72fcb47d741ed81b208d96c5a2c5d%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637659956810925035%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0%2Bt7uIOrBYc749Mz6gxT%2B5OiFuq5qDfvGwxfpxA7rJc%3D&reserved=0>
> > UCL SHaPS
> > Chandler House
> > 2 Wakefield Street
> > London WC1N 1PF
> >
> > Personal website
> > <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fclarelally.wordpress.com%2F&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7Cb0c72fcb47d741ed81b208d96c5a2c5d%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637659956810925035%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=R8gO5pRFDuStPFuNuTV0rzDf1mMoJiHudgJcO4W4US0%3D&reserved=0> | Twitter:
> > @Clare_Lally <https://twitter.com/Clare_Lally>
> >
>
> --
> Guillaume Flandin, PhD
> Wellcome Centre for Human Neuroimaging
> UCL Queen Square Institute of Neurology
> London WC1N 3BG
>
>
--
Guillaume Flandin, PhD
Wellcome Centre for Human Neuroimaging
UCL Queen Square Institute of Neurology
London WC1N 3BG
|