Dear Guillaume,
Thank you for your response! That's right about the data being over 7000 scans. The design is less neat... I am using open-access pre-processed data from the Naturalistic Neuroimaging Database, where participants were scanned whilst they watched a feature-length movie. The events that I am modelling are instances of particular movie characters speaking, so the 'trials' are unequal sizes and distributed differently across the run.
I will try running one participant at a time, and I will try to keep note of when the script crashes.
I remember you mentioned this issue with disabling masking before, I use a mask when I conduct the next stage of my analyses (multivariate) but it sounds like I need to do something at this stage to. My SPM script produces the binary mask.nii and the output is sensible. Can I apply this mask at this stage? Alternatively I could exclude any values that are zero exactly (as all values outside of the brain are zero) although I worry I'd accidentally exclude values inside the brain as well. Which would you advise and please could I have some guidance on how to include this in the code? Thank you!!!
All the best,
Clare
On 27/08/2021, 10:49, "Flandin, Guillaume" <[log in to unmask]> wrote:
Dear Clare,
A two hour recording in a single session is long. Do I understand it
correctly that there is on average one trial every 30 seconds and that
they are spread across 24 conditions? With a TR of 1s, this means that
you have over 7000 scans?
While it should not happen, I presume that the error is due to out of
memory when handling covariance matrices. Do you know at which stage do
the crashes occur? Not a satisfactory answer but it might help if you
close and restart MATLAB in between the analysis of each subject (there
is also a 'pack' command in MATLAB but I don't know how well it would
work in your case).
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mathworks.com%2Fhelp%2Fmatlab%2Fref%2Fpack.html&data=04%7C01%7Cc.lally%40UCL.AC.UK%7Cd8a34ee2e3b045b4769508d9693ff293%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637656545633386684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Mq8Pg5dyMIqdhzeRbHkLrIb6fVET9k4mEPl5X1tDS7M%3D&reserved=0
One thing I notice in your script is that you disabled masking:
jobs{1}.stats{1}.fmri_spec.mthresh = -Inf;
jobs{1}.stats{1}.fmri_spec.mask = {''};
From the comment in your code, I understand that the data have already
been mean-centred during preprocessing hence the change but make sure
outside of the brain is not considered during the GLM analysis otherwise
provide an explicit mask.
Best regards,
Guillaume.
On 26/08/2021 11:26, Lally, Clare wrote:
> Hi everyone,
>
>
>
> Please could I have some advice on how to make my first level model
> estimation batch script more computationally efficient? I am working
> with a large dataset, in which participants have 2 hours of functional
> data in a single run. In my model, I have 242 events distributed across
> 24 conditions.
>
>
>
> I have been running the script below on a 3.6 GHz 16 GB iMac, but it is
> taking over 8 hours per participant. Also, the script routinely crashes
> with the generic ‘MATLAB has encountered an error and needs to close’,
> which means I have to restart the script from the current participant.
> However, it does run the whole way through otherwise.
>
>
>
> I suspect the script runs slow and crashes because the code is producing
> things that don’t need to be created in the first place, or that can be
> removed from memory once they have served their purpose.
>
>
>
> I’d be very grateful for any advice on how to check this or what to
> change. Thank you!
>
>
>
> Clare
>
>
>
>
>
> %% Univariate matrix items vs. baseline - estimate the model
>
>
>
> % Specify whether to conduct model estimation (time-consuming)
>
> ESTIMATE_GLM = 1; % Set this to zero to check the script without
> estimating
>
>
>
> % Scanning/timing details
>
> TR = 1;
>
> timings = readtable(fullfile(timings_path, 'Contrasts_file.csv'));
>
> runs = unique(timings.Run);
>
> conditions = unique(timings.ConditionName);
>
> runs_n = length(runs);
>
> conditions_n = length(conditions);
>
> onsets_n = length(timings.Onset);
>
>
>
> % Start for loop for first level analyses here
>
> for p = 1:numel(ppt)
>
>
>
> % Progress message
>
> disp(['Processing ', char(ppt(p)), ' ...'])
>
>
>
> % Define locations for data and outputs
>
> func_path = fullfile(data_path, char(ppt(p)), 'func');
>
> funcs = dir(fullfile(data_path, char(ppt(p)), 'func',
> [char(ppt(p)) '*_bold_no_blur_no_censor.nii']));
>
> funcs = funcs.name;
>
> numScans = numel(spm_vol(fullfile(func_path,funcs)));
>
> anat = fullfile(data_path, char(ppt(p)), 'anat',
> [char(ppt(p)) '_t1w_mni_alignment.nii']);
>
> output_path = fullfile(data_path, char(ppt(p)), output_folder);
>
>
>
> % Begin creating jobs structure
>
> jobs{1}.stats{1}.fmri_spec.dir = cellstr(output_path);
>
> jobs{1}.stats{1}.fmri_spec.timing.units = 'secs';
>
> jobs{1}.stats{1}.fmri_spec.timing.RT = TR;
>
> jobs{1}.stats{1}.fmri_spec.timing.fmri_t = 16;
>
> jobs{1}.stats{1}.fmri_spec.timing.fmri_t0 = 1;
>
>
>
> % Create multiple conditions .mat file for each run
>
> for r = 1:size(runs_n, 1)
>
> % Create empty structures for onsets and durations
>
> names = conditions';
>
> onsets = cell(1, size(conditions,1));
>
> durations = cell(1, size(conditions,1));
>
>
>
> % For each condition
>
> for c = 1:size(conditions,1)
>
>
>
> % For each onset of an event
>
> for x = 1:onsets_n
>
> if isequal(timings.ConditionName(x), conditions(c)) &&
> timings.Run(x) == r
>
> onsets{c} = double([onsets{c} timings.Onset(x)]);
>
> durations{c} = double([durations{c}
> timings.Duration(x)]);
>
>
>
> end
>
> end
>
> end
>
>
>
> % Save the structure as a .mat file for each run per participant
>
> save (fullfile(output_path, [mat_prefix, char(ppt(p)), '_run',
> num2str(runs_n(r))]), ...
>
> 'names', 'onsets', 'durations')
>
>
>
> % Get frames for each run using spm_select - add run-related
> file indexing to the line below if needed
>
> files = spm_select('ExtFPList', func_path, funcs, 1:numScans);
>
>
>
> % Fill in session information within jobs structure
>
> jobs{1}.stats{1}.fmri_spec.sess(r).scans = cellstr(files);
>
> jobs{1}.stats{1}.fmri_spec.sess(r).cond = struct('name',
> {}, 'onset', {}, 'duration', {}, 'tmod', {}, 'pmod', {});
>
> jobs{1}.stats{1}.fmri_spec.sess(r).multi =
> cellstr(fullfile(output_path, [mat_prefix, char(ppt(p)), '_run',
> num2str(runs_n(r)), '.mat']));
>
> jobs{1}.stats{1}.fmri_spec.sess(r).regress = struct('name',
> {}, 'val', {});
>
> jobs{1}.stats{1}.fmri_spec.sess(r).multi_reg = {''};
>
> jobs{1}.stats{1}.fmri_spec.sess(r).hpf = 128;
>
> end
>
>
>
> % Fill in the rest of the jobs fields
>
> jobs{1}.stats{1}.fmri_spec.fact = struct('name', {}, 'levels', {});
>
> jobs{1}.stats{1}.fmri_spec.bases.hrf = struct('derivs', [0 0]);
>
> jobs{1}.stats{1}.fmri_spec.volt = 1;
>
> jobs{1}.stats{1}.fmri_spec.global = 'None';
>
> jobs{1}.stats{1}.fmri_spec.mthresh = -Inf; % For mean-centred
> functional images - needed for NNDb
>
> jobs{1}.stats{1}.fmri_spec.mask = {''};
>
> jobs{1}.stats{1}.fmri_spec.cvi = 'AR(1)';
>
>
>
> % Navigate to output directory, specify and estimate GLM
>
> cd(output_path);
>
> spm_jobman('run', jobs)
>
>
>
> if ESTIMATE_GLM == 1
>
> load SPM;
>
> spm_spm(SPM);
>
> end
>
> end
>
>
>
> Clare Lally (Post-doctoral Research Fellow)
> UCL Vocal Communication Laboratory
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.carolynmcgettigan.com%2F&data=04%7C01%7Cc.lally%40UCL.AC.UK%7Cd8a34ee2e3b045b4769508d9693ff293%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637656545633386684%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=z%2Fz9sxRW2bKcTI7fTg4kvvcjWoC0qPtA8huj8cToHT0%3D&reserved=0>
> UCL SHaPS
> Chandler House
> 2 Wakefield Street
> London WC1N 1PF
>
> Personal website
> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fclarelally.wordpress.com%2F&data=04%7C01%7Cc.lally%40UCL.AC.UK%7Cd8a34ee2e3b045b4769508d9693ff293%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637656545633396677%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VO6xlnXVXy6dSIJvS0MpuqvzjuPiZCuzb0F699kRqpQ%3D&reserved=0> | Twitter:
> @Clare_Lally <https://twitter.com/Clare_Lally>
>
--
Guillaume Flandin, PhD
Wellcome Centre for Human Neuroimaging
UCL Queen Square Institute of Neurology
London WC1N 3BG
|
