I thought that the number should be 7 to 11 for the Diamond cluster. Something to discuss tomorrow.
Just so you know, the default in XCE, when you open it for the first time, is 7.
A wrong number can be entered when you are using NX and typing something, thinking you are typing "somewhere else" but you are actually typing in XCE, in the random cell that have been selected.
It happened to me.
(Sent from a tiny little phone)
Le 7 mai 2019 à 22:02, Frank von Delft <[log in to unmask]<mailto:[log in to unmask]>> a écrit :
Is Romain's info up-to-date? I thought Rachael made it some high cpu number to force the queue to send it only to high memory nodes.
If not: how would the "wrong" number get in there?
-------- Forwarded Message --------
Subject: Re: [XChemBB] Pandda memory allocation error
Date: Tue, 7 May 2019 15:27:50 +0000
From: [log in to unmask]<mailto:[log in to unmask]> <[log in to unmask]><mailto:[log in to unmask]>
Reply-To: [log in to unmask]<mailto:[log in to unmask]> <[log in to unmask]><mailto:[log in to unmask]>
To: [log in to unmask]<mailto:[log in to unmask]>
Hi Sabrina,
For some reason you had cpus=47 in the pandda.sh script file generated by XCE.
It should be cpus=7 (default).
This is what caused the issue I believe.
Let see: I started your pandda job and it is currently running.
If you want to see if pandda is still running, either type in a terminal window once in your project folder:
tail analysis/panddas/logs/*log (it will display the last rows of the pandda log) or
qstat –u mlk83353.
Cheers,
Romain.
From: XChem bulletin board <[log in to unmask]><mailto:[log in to unmask]> On Behalf Of [log in to unmask]<mailto:[log in to unmask]>
Sent: 07 May 2019 15:58
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [XChemBB] Pandda memory allocation error
Hi Sabrina,
Could you send me – in a separate message if you prefer – your visit path?
I have recently run pandda jobs for an old project which has >1500 datasets.
I had issues coming from mismatching panda versions (between the existing old pandda data and the new run) so I had to re-run pandda in a new “panddas” folder.
Could it be the case here? If you backup your existing panddas folder and run it fresh in a new one, does it run?
Cheers,
Romain.
From: XChem bulletin board <[log in to unmask]<mailto:[log in to unmask]>> On Behalf Of Conor Wild
Sent: 06 May 2019 09:21
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [XChemBB] Pandda memory allocation error
So I took a quick look through the log. There isn't any obvious warning signs that I could see.
As for things I need, it is all in the log file (@Elliot just look in settings different to default), so if you're happy I can try it in my environment. Elliot also looks like hes volunteering himself to check it on a different queue.
Thanks, Conor
On Mon, 6 May 2019 at 08:04, Elliot Nelson <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Hi conor/sabrina,
Conor, i presume you can run this on medium queue, and specify extra RAM.
Sabrina: If you can give us the path to your folder we can give it a go. Is there are any special setup for the run?
Elliot
________________________________
From: XChem bulletin board <[log in to unmask]<mailto:[log in to unmask]>> on behalf of Sabrina Mackinnon <[log in to unmask]<mailto:[log in to unmask]>>
Sent: 04 May 2019 23:49:01
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [XChemBB] Pandda memory allocation error
Hi Connor,
Thanks for your reply - to clarify, I am running on the Diamond servers through NoMachine. Do you think re-running it on your setup would help?
Thanks,
Sabrina
Sabrina Mackinnon AMRSB
D. Phil Student in Metabolic & Rare Diseases
Structural Genomic Consortium
Nuffield Department of Medicine
University of Oxford, OX3 7DQ
________________________________
From: Conor Wild <[log in to unmask]<mailto:[log in to unmask]>>
Sent: 04 May 2019 23:06:41
To: Sabrina Mackinnon
Cc: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [XChemBB] Pandda memory allocation error
Hey Sabrina,
I have my own PanDDA setup - if you'd like I could try running the analysis with it on the Diamond servers?
Might help debug whether the cause of the problem is environment related?
Thanks, Conor
On Sat, 4 May 2019 at 22:31, Sabrina Mackinnon <[log in to unmask]<mailto:[log in to unmask]>> wrote:
I have just tried starting over - re-import auto-processing, run DIMPLE, run pandda - and I get a brand new error "SystemError: NULL result without error in PyObject_Call". I can't find any mention of this in the archives - anyone recognise this?
Cheers,
Sabrina
Traceback (most recent call last):
File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/__init__.py", line 1081, in run
pandda_analyse_main(pandda=pandda)
File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/__init__.py", line 1039, in pandda_analyse_main
pandda_main_loop(pandda=pandda)
File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/__init__.py", line 388, in pandda_main_loop
map_resolution = cut_resolution)
File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/classes.py", line 2251, in load_and_morph_maps
dataset_maps = easy_mp.pool_map(func=wrapper_run, args=arg_list, processes=self.settings.cpus, chunksize=1)
File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/py2/site-packages/cctbx_project/libtbx/easy_mp.py", line 408, in pool_map
result = pool.map(func=func, iterable=iterable, chunksize=chunksize)
File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/multiprocessing/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
SystemError: NULL result without error in PyObject_Call
Sabrina Mackinnon AMRSB
D. Phil Student in Metabolic & Rare Diseases
Structural Genomic Consortium
Nuffield Department of Medicine
University of Oxford, OX3 7DQ
________________________________
From: Pearce, N.M. (Nick) <[log in to unmask]<mailto:[log in to unmask]>>
Sent: 04 May 2019 13:38:35
To: Sabrina Mackinnon
Cc: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [XChemBB] Pandda memory allocation error
No — XCE should be doing it right. I’ll let someone from diamond chip in. They might be able to help. But I see no reason it shouldn’t be working.
Thanks,
Nick
On 4 May 2019, at 14:18, Sabrina Mackinnon <[log in to unmask]<mailto:[log in to unmask]>> wrote:
I am running through XCE. Should I try from the command line?
Thanks,
Sabrina
Sabrina Mackinnon AMRSB
D. Phil Student in Metabolic & Rare Diseases
Structural Genomic Consortium
Nuffield Department of Medicine
University of Oxford, OX3 7DQ
________________________________
From: Pearce, N.M. (Nick) <[log in to unmask]<mailto:[log in to unmask]>>
Sent: 04 May 2019 08:23:59
To: Sabrina Mackinnon
Cc: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [XChemBB] Pandda memory allocation error
Oh — yes that changes everything. I’ve cc’d the BB back in since this is diamond related. Are you running from the command line or through XCE?
Thanks,
Nick
On 3 May 2019, at 23:41, Sabrina Mackinnon <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Hi Nick,
Sorry I was unclear - I am running on the diamond cluster through NoMachine - does that help?
Cheers,
Sabrina
Sabrina Mackinnon AMRSB
D. Phil Student in Metabolic & Rare Diseases
Structural Genomic Consortium
Nuffield Department of Medicine
University of Oxford, OX3 7DQ
________________________________
From: Pearce, N.M. (Nick) <[log in to unmask]<mailto:[log in to unmask]>>
Sent: 03 May 2019 21:56:31
To: Sabrina Mackinnon
Subject: Re: [XChemBB] Pandda memory allocation error
It’s not really designed to be run on a normal desktop — and if you’re using a normal SGC desktop I wouldn’t expect much. Can’t say more without actual specifications though.
Thanks,
Nick
On 3 May 2019, at 19:59, Sabrina Mackinnon <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Hi Nick,
Thanks for getting back to me so quickly.
I have tried running it both from my home PC and SGC desktop so I'm fairly sure it's not the computer. It's 1 molecule of ~45kDa in the asymmetric unit.
Thanks,
Sabrina
Sabrina Mackinnon AMRSB
D. Phil Student in Metabolic & Rare Diseases
Structural Genomic Consortium
Nuffield Department of Medicine
University of Oxford, OX3 7DQ
________________________________
From: Pearce, N.M. (Nick) <[log in to unmask]<mailto:[log in to unmask]>>
Sent: 03 May 2019 18:25:47
To: Sabrina Mackinnon
Cc: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: [XChemBB] Pandda memory allocation error
What are the specifications of the computer you’re running on and what is the number of residues in the ASU?
Thanks,
Nick
> On 3 May 2019, at 18:40, Sabrina Mackinnon <[log in to unmask]<mailto:[log in to unmask]>> wrote:
>
> Hi,
>
> I am trying to run pandda on a set of 113 datasets but after ~20 minutes it errors with the message "OSError: [Errno 12] Cannot allocate memory".
>
> I can see from earlier posts back in 2016 that this can occur with a large number of datasets (>1000) or where the unit cell is very big (mine is only 217 217 217 90 90 90) but neither of these issues seems to apply here.
>
> Does anyone know what else could be causing this issue and/or how to fix it?
>
> Full trace back is:
>
> Traceback (most recent call last):
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/__init__.py", line 1081, in run
> pandda_analyse_main(pandda=pandda)
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/__init__.py", line 1039, in pandda_analyse_main
> pandda_main_loop(pandda=pandda)
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/__init__.py", line 388, in pandda_main_loop
> map_resolution = cut_resolution)
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/site-packages/pandda/analyse/classes.py", line 2251, in load_and_morph_maps
> dataset_maps = easy_mp.pool_map(func=wrapper_run, args=arg_list, processes=self.settings.cpus, chunksize=1)
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/py2/site-packages/cctbx_project/libtbx/easy_mp.py", line 403, in pool_map
> fixed_func=fixed_func)
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/py2/site-packages/cctbx_project/libtbx/easy_mp.py", line 135, in __init__
> initargs=initargs)
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/multiprocessing/pool.py", line 159, in __init__
> self._repopulate_pool()
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/multiprocessing/pool.py", line 223, in _repopulate_pool
> w.start()
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/multiprocessing/process.py", line 130, in start
> self._popen = Popen(self)
> File "/dls/science/groups/i04-1/software/pandda_0.2.12/ccp4/ccp4-7.0/lib/python2.7/multiprocessing/forking.py", line 121, in __init__
> self.pid = os.fork()
>
> Many thanks,
>
> Sabrina
>
> ########################################################################
>
> To unsubscribe from the XCHEMBB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMBB&A=1
________________________________
To unsubscribe from the XCHEMBB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMBB&A=1
________________________________
To unsubscribe from the XCHEMBB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMBB&A=1
________________________________
To unsubscribe from the XCHEMBB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMBB&A=1
________________________________
To unsubscribe from the XCHEMBB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMBB&A=1
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
________________________________
To unsubscribe from the XCHEMBB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMBB&A=1
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
________________________________
To unsubscribe from the XCHEMBB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMBB&A=1
########################################################################
To unsubscribe from the XCHEMDEV list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=XCHEMDEV&A=1
|
