Hi Kelly,
It seems the tail approximation (that happens after permutations reach
100%) is taking too long, then hitting the wall time limit. The 21000
seconds are less than 6 hours, so all the other 144 hours (!) are somehow
being spent on the tail approximation, which doesn't make sense. Consider
changing "-approx tail" for "-approx gamma" (keep the "-nouncorrected").
It's a different approximation but tends not to differ too much.
All the best,
Anderson
On Tue, 28 Jan 2020 at 16:04, Hiersche, Kelly J. <[log in to unmask]> wrote:
> Hello,
>
> The log file is frozen at saving p-values. All the permutations have run
> and then I get:
>
> ...
> 100% [Design 1/1, Contrast 2/2, Shuffling 5000/5000, Modality 1/1]
> Elapsed time with permutations: ~ 20929.9 seconds.
> Computing p-values. Saving p-values (uncorrected, and corrected within
> modality and within contrast). =>> PBS: job killed: walltime 540037
> exceeded limit 540000
>
> I am running the analysis on the university supercomputer. I feel like it
> might be stuck on an error, but no error message has appeared on any of the
> runs I have tried. I keep raising the walltime, but even with 150 hours,
> PALM did not finish running. I can try running it using more processors per
> node, I just wanted to make sure that it makes sense for this to be talking
> so long.
>
> Thank you,
>
> Kelly
> ------------------------------
> *From:* FSL - FMRIB's Software Library <[log in to unmask]> on behalf of
> Anderson M. Winkler <[log in to unmask]>
> *Sent:* Monday, January 27, 2020 6:07 PM
> *To:* [log in to unmask] <[log in to unmask]>
> *Subject:* Re: [FSL] PALM issue saving p-values
>
> Hi Kelly,
>
> Either PALM is still running (it seems it is from your message) or it has
> crashed. These tstat files are saved at the beginning as they don't need
> permutations, but the p-values can only be produced at the far end.
>
> If you don't have access to the console where it's running (say, because
> it's in a cluster and somehow you can see the log files), then you can
> check that it has finished successfully if the files
> results_1_21_2020_elapsed.csv have been produced.
>
> Hope this helps!
>
> All the best,
>
> Anderson
>
> On Mon, 27 Jan 2020 at 14:46, Hiersche, Kelly J. <[log in to unmask]>
> wrote:
>
> Thank you Paul,
>
> I tried specifying the -ise to run sign flipping. I was able to run PALM
> using the settings:
> -i /fs/project/hcppalm/task_01_13_2020/Cope10combinednew.dtseries.nii
> -transposedata
> -d /fs/project/hcppalm/task_01_13_2020/designmatrix.mat
> -t /fs/project/hcppalm/task_01_13_2020/C.con
> -eb EBnew.csv -ise
> -o /fs/project/hcppalm/task_1_21_2020
> -n 5000 -corrcon -logp -accel tail -nouncorrected
>
> I have gotten a couple outputs:
> results_1_21_2020_dat_tstat_c1.dscalar.nii
> results_1_21_2020_dat_tstat_c2.dscalar.nii
>
> I believe I should be getting more outputs than this. According to the
> tutorial I should get a result with the p-values FWER and the p-values FWER
> across all contrasts from all designs, but PALM has been running for 150
> hours (for ~700 subjects, 2 contrasts) and I have not gotten the p-value
> results.
>
> Any ideas?
>
>
> ------------------------------
> *From:* FSL - FMRIB's Software Library <[log in to unmask]> on behalf of
> Paul Zhutovsky <[log in to unmask]>
> *Sent:* Monday, January 27, 2020 7:08 AM
> *To:* [log in to unmask] <[log in to unmask]>
> *Subject:* Re: [FSL] PALM issue saving p-values
>
> Hi,
>
> I think you want to run a sign-flipping test and not a permutation test.
> For this you would need to specify the '-ise' option in PALM. See this
> example
> <https://urldefense.com/v3/__https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/PALM/Examples*Example_2:_Permutations_with_sign-flipping__;Iw!!KGKeukY!g3ULJrvriLwfHTEuSTGDbBRM-5IeWR08mE1HwlLgZitSW4OvjzokTKNA1WlRsgFI$>
> .
>
> Cheers,
> Paul
>
> On Thu, Jan 23, 2020 at 9:03 PM Hiersche, Kelly J. <[log in to unmask]>
> wrote:
>
> Hello,
>
> I do have a column of just ones for my design matrix, which I thought was
> to represent the input COPE image, so the first four columns are
> information such as age, gender, education, etc. Then the fifth column is a
> column of ones for the COPE input image.
>
> I have attached the my con matrix. I am not attaching the design, as it
> has restricted data, but I am attaching a sample with only 5 subjects. My
> actual design matrix is similar, only with over 700 subjects.
>
> Thank for the help!
>
> Kelly
>
> ------------------------------
> *From:* FSL - FMRIB's Software Library <[log in to unmask]> on behalf of
> Colin Hawco <[log in to unmask]>
> *Sent:* Monday, January 13, 2020 3:22 PM
> *To:* [log in to unmask] <[log in to unmask]>
> *Subject:* Re: [FSL] PALM issue saving p-values
>
> The key things here, I think:
>
> Generating 1 shufflings (permutations only).
>
>
> For whatever reason PALM can only make 1 permutation. For example, you
> gave it a column of 1's in the design matrix, and it is trying to run
> shuffling instead of sign flipping.
>
> Maybe paste in your design and con matricies.
>
> Colin
>
>
> Quoting Kelly Hiersche <[log in to unmask]>:
>
> > I am having the same problem while running PALM. I get the following
> output:
> >
> > Running PALM alpha116 using MATLAB 9.7.0.1190202 (R2019b) with the
> > following options:
> > -i Cope10combinednew.dtseries.nii
> > -transposedata
> > -d designmatrix.mat
> > -t C.con
> > -eb EBnew.csv
> > -o results
> > -n 5000
> > -corrcon
> > -logp
> > -accel tail
> > -nouncorrected
> > Found HCP Workbench executable in
> > /usr/local/connectome-workbench/1.3.2/bin_rh_linux64/wb_command
> > Reading input 1/1:
> >
> /fs/project/PAS1418/hcppalm/motor_01_13_2020/Cope10combinednew.dtseries.nii
> > Reading design matrix and contrasts.
> > Elapsed time parsing inputs: ~ 7.05839 seconds.
> > Number of possible permutations is 1.
> > Generating 1 shufflings (permutations only).
> > Building null distribution.
> > 50% [Design 1/1, Contrast 1/2, Shuffling 1/1, Modality 1/1]
> > Saving file: /fs/project/PAS1418/hcppalm/results_dat_tstat_c1
> > Number of possible permutations is 1.
> > Generating 1 shufflings (permutations only).
> > Building null distribution.
> > 100% [Design 1/1, Contrast 2/2, Shuffling 1/1, Modality 1/1]
> > Saving file: results_dat_tstat_c2
> > Elapsed time with permutations: ~ 10.6722 seconds.
> > Computing p-values.
> > Saving p-values (uncorrected, and corrected within modality and
> > within contrast).
> > { Unable to perform assignment because the size of the left side is
> > 1-by-1 and
> > the size of the right side is 0-by-1.
> >
> > Error in palm_competitive (line 96)
> > S(infpos,c) = max(S(~infpos,c)) + 1;
> >
> > Error in palm_datapval (line 67)
> > [~,cdfG,distp] = palm_competitive(Gvals(:),'descend',true);
> >
> > Error in palm_pareto (line 78)
> > P = palm_datapval(G,Gdist,rev);
> >
> > Error in palm_saveall (line 246)
> > Ptosave =
> >
> >
> palm_pareto(plm.G{y}{m}{c},plm.Gmax{y}{m}{c},false,opts.accel.tail_thr,opts.accel.G1out);
> >
> > Error in palm_core (line 2489)
> > palm_saveall(plm,opts);
> >
> > Error in palm (line 81)
> > palm_core(varargin{:});
> > }
> >
> >
> > I am working with the HCP data (restricted and unrestricted).
> >
> > Any suggestions/help would be appreciated!
> >
> > Thanks,
> >
> > Kelly
> >
> > ########################################################################
> >
> > To unsubscribe from the FSL list, click the following link:
> >
> https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1__;!!KGKeukY!n2unrmGioD_x7P_9DZb89w_FV7d3Eh9GywS7BueE-t7czu1VQAjbttPdAPOEC4B5$
>
> ########################################################################
>
> To unsubscribe from the FSL list, click the following link:
>
> https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1__;!!KGKeukY!n2unrmGioD_x7P_9DZb89w_FV7d3Eh9GywS7BueE-t7czu1VQAjbttPdAPOEC4B5$
>
> ------------------------------
>
> To unsubscribe from the FSL list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1
> <https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1__;!!KGKeukY!g3ULJrvriLwfHTEuSTGDbBRM-5IeWR08mE1HwlLgZitSW4OvjzokTKNA1Y7g2DMM$>
>
>
> ------------------------------
>
> To unsubscribe from the FSL list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1
> <https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1__;!!KGKeukY!g3ULJrvriLwfHTEuSTGDbBRM-5IeWR08mE1HwlLgZitSW4OvjzokTKNA1Y7g2DMM$>
>
> ------------------------------
>
> To unsubscribe from the FSL list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1
> <https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1__;!!KGKeukY!lgD5ZZHu2vb_zBTNeox-PUvrkxAhKGT-8luFzb1s6wwVSoU9leWN-Y_mBm9HRSyO$>
>
>
> ------------------------------
>
> To unsubscribe from the FSL list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1
> <https://urldefense.com/v3/__https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1__;!!KGKeukY!lgD5ZZHu2vb_zBTNeox-PUvrkxAhKGT-8luFzb1s6wwVSoU9leWN-Y_mBm9HRSyO$>
>
########################################################################
To unsubscribe from the FSL list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=FSL&A=1
