JiscMail Logo
Email discussion lists for the UK Education and Research communities

Help for CCPEM Archives


CCPEM Archives

CCPEM Archives


CCPEM@JISCMAIL.AC.UK


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

CCPEM Home

CCPEM Home

CCPEM  September 2018

CCPEM September 2018

Options

Subscribe or Unsubscribe

Subscribe or Unsubscribe

Log In

Log In

Get Password

Get Password

Subject:

Re: Empty logfile.pdf with Relion 3 on the tutorial dataset

From:

Takanori Nakane <[log in to unmask]>

Reply-To:

Takanori Nakane <[log in to unmask]>

Date:

Fri, 14 Sep 2018 17:42:57 +0100

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (215 lines)

Hi,

This sounds like a file system issue. That is, ghostscript
does not 'see' the EPS files RELION just created, for some reason.

In past, we had a bug where multiple threads of CtfRefine wrote to
a same file, sometimes corrupting the output. This bug was fixed on July 9th,
one month before we went to public beta testing.

> * when the job fails, if we tried to use the gs command to convert the
> intermediate eps files to pdf, it works

This means EPS outputs are fine.
If EPS files are broken, ghostscript complains.

> * The post-processing job remains stuck in the "running job" section of
> the GUI and no logfile.pdf is produced either.

The GUI thinks the job is still running because logfile.pdf is not there.
Since post-processing does not use multi-threading, this also excludes the
possibility of the above mentioned bug.

Best regards,

Takanori Nakane

> Hi -
>
> Thanks a lot for the feedback, Sjors - We have made some more tests and
> these are the observations:
>
> * the MotionCor2 jobs sometimes fails to produce the final logfile.pdf:
> the file exists but is empty or very small (a few kb, and cannot be
> opened; "The file has been damaged" message obtained)
>
> * When the generation of the logfile fails, no error message is seen in
> either run.out or run.err. The run.out still ends with this message
>
> " Generating logfile.pdf ...
>
>    0/   0 sec
> ............................................................~~(,_,">
>  Done! Written: MotionCorr/job055/logfile.pdf and
> MotionCorr/job055/corrected_micrographs.star
> Program finished with exit code 0 at: Thu Sep 13 16:17:45 EDT 2018"
>
> * We tried on different nodes (we have some with 4 or 8 gpus). For some
> reasons, some of the jobs do generate the logfile.pdf properly (it seems
> random, though it tends to fail more on nodes with 4 gpus). In all cases,
> if we check the intermediate eps files, they look the same and correct.
>
> * when the job fails, if we tried to use the gs command to convert the
> intermediate eps files to pdf, it works
>
> * if we run the jobs directly from the command line as suggested, the
> logfile is created properly (command line corresponding to the job that
> failed when submitted to the queue and by logging into the same node
> before running the command)
>
>
> Other behavior we have observed and which might be related:
>
> * The post-processing job remains stuck in the "running job" section of
> the GUI and no logfile.pdf is produced either. There is no error message
> seen in run.err and run.out ends with:
>
> "  + FINAL RESOLUTION:              3.62045
>
> Program finished with exit code 0 at: Fri Sep 14 11:25:42 EDT 2018"
>
> When I check the list of jobs running on the cluster, the post-processing
> job ID cannot be seen and if I log into the node, no relion job is running
> anymore.
>
>
> We're using gs 9.07, if that matters.
>
>
> If anyone has any idea, thanks for letting us know.
>
>
> Thanks,
>
> Best,
>
> Nicolas
>
>
>
> ________________________________
> From: Sjors Scheres <[log in to unmask]>
> Sent: Saturday, September 8, 2018 1:00 PM
> To: Coudray, Nicolas
> Cc: [log in to unmask]
> Subject: Re: [ccpem] Empty logfile.pdf with Relion 3 on the tutorial
> dataset
>
> Hi Nicolas,
> RELION will make a system call to "gs" to combine the EPS files into a
> single logfile.pdf. If it fails, it should write to stderr. Did you see
> such an error in the run.err of those directories? You could try executing
> the command from the command line to find out what the problem on your
> system is.
> HTH,
> Sjors
>
> PS: I am pasting the relevant code from src/CPlot2D.cpp in case it helps
> below:
>
>
>
>     FileName fn_list = fn_pdf + ".lst";
>     std::string command = "gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER
> -dDEVICEWIDTHPOINTS=800 -dDEVICEHEIGHTPOINTS=800 -sOutputFile=";
>     command += fn_pdf + " @" + fn_list;
>     std::ofstream filelist(fn_pdf + ".lst");
>     bool have_at_least_one = false;
>     for (int i = 0; i < fn_eps.size(); i++)
>     {
>         // fn_eps[i] could be a Linux wildcard...
>         std::vector<FileName> all_eps_files;
>         fn_eps[i].globFiles(all_eps_files);
>         fn_eps[i].globFiles(all_eps_files);
>         for (long int j= 0; j < all_eps_files.size(); j++)
>         {
>                 if (exists(all_eps_files[j]))
>                 {
>                         filelist << all_eps_files[j] << "\n";
>                         have_at_least_one = true;
>                 }
>         }
>     }
>     filelist.close();
>
>
>
>> Hi -
>>
>>
>>  We are currently testing Relion 3 and have some weird behavior with the
>> pdf files generated during the motion-correction  they are empty.
>>
>>
>>   We are using the tutorial dataset and we ran several very similar
>> Motion
>> Correction jobs; 11 out of 15 runs gave empty "logfile.pdf" file (it
>> seems to not depend on the number of GPU or CPUs - it happens for both
>> motioncor and "relioncor" - main parameters in the I/O and Motion tabs
>> were not changed). Also, the eps files of the movies all seem ok though
>> (non-empty, can be converted to pdf with ps2pdf and can be open
>> properly) and none of the run  generate any error message.
>>
>>
>>   Does anyone know how to solve this issue or how we could track down
>> its
>> origin?
>>
>>
>> Thanks,
>>
>> Best,
>>
>> Nicolas
>>
>> ------------------------------------------------------------
>> This email message, including any attachments, is for the sole use of
>> the
>> intended recipient(s) and may contain information that is proprietary,
>> confidential, and exempt from disclosure under applicable law. Any
>> unauthorized review, use, disclosure, or distribution is prohibited. If
>> you have received this email in error please notify the sender by return
>> email and delete the original message. Please note, the recipient should
>> check this email and any attachments for the presence of viruses. The
>> organization accepts no liability for any damage caused by any virus
>> transmitted by this email.
>> =================================
>>
>> ########################################################################
>>
>> To unsubscribe from the CCPEM list, click the following link:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCPEM-26A-3D1&d=DwIDAg&c=j5oPpO0eBH1iio48DtsedbOBGmuw5jHLjgvtN2r4ehE&r=RRJopfWEMpjEmb1q4aEqz5Y4zjDaSRnr5kurG-GlA7g&m=YpneSos6s6bBDAyHgYauYwKyJrAKiYvYgRFdrfD6G1s&s=EXXDqcXpySIlYfZ9IR88mVlQSU0D7Q6l3lTgGc0ShZ4&e=
>>
>
>
> --
> Sjors Scheres
> MRC Laboratory of Molecular Biology
> Francis Crick Avenue, Cambridge Biomedical Campus
> Cambridge CB2 0QH, U.K.
> tel: +44 (0)1223 267061
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www2.mrc-2Dlmb.cam.ac.uk_groups_scheres&d=DwIDAg&c=j5oPpO0eBH1iio48DtsedbOBGmuw5jHLjgvtN2r4ehE&r=RRJopfWEMpjEmb1q4aEqz5Y4zjDaSRnr5kurG-GlA7g&m=YpneSos6s6bBDAyHgYauYwKyJrAKiYvYgRFdrfD6G1s&s=sjEthfBCV6DItq1Iv_xaDpwaNkDcXKZ36-OzXzijo_0&e=
>
>
> ------------------------------------------------------------
> This email message, including any attachments, is for the sole use of the
> intended recipient(s) and may contain information that is proprietary,
> confidential, and exempt from disclosure under applicable law. Any
> unauthorized review, use, disclosure, or distribution is prohibited. If
> you have received this email in error please notify the sender by return
> email and delete the original message. Please note, the recipient should
> check this email and any attachments for the presence of viruses. The
> organization accepts no liability for any damage caused by any virus
> transmitted by this email.
> =================================
>
> ########################################################################
>
> To unsubscribe from the CCPEM list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
>

########################################################################

To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1

Top of Message | Previous Page | Permalink

JiscMail Tools


RSS Feeds and Sharing


Advanced Options


Archives

September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013


JiscMail is a Jisc service.

View our service policies at https://www.jiscmail.ac.uk/policyandsecurity/ and Jisc's privacy policy at https://www.jisc.ac.uk/website/privacy-notice

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager