Dear Yangqi,
Yes, RAM is the most likely answer given that you managed to get it to work for smaller data sets. K-means for large datasets requires large memory capacity. One option is to spread it across multiple nodes to make use of more total memory. Another option is to go down in the number of CPUs for this 16 core machine as Spring's multi-CPU jobs (Openmpi) memory will not be shared within a machine. If you use less CPUs you may need less total memory on your machine but it will be slower in the alignment steps.
Best wishes,
Carsten
> On Dec 6, 2018, at 21:11, Yangqi Gu <[log in to unmask]> wrote:
>
> Dear Spring developers,
> I am trying to do a segment classification on my helical particles. I have ~15,000 particles with the box size of 384 A and I tried using 5 iterations for 10 classes. I used MPI options with 16 (Since I have 16 cores and 64G RAM in total). However, it always failed after (frozen) 1 or 2 iterations with some unknown reasons. (When I used a smaller datasets, ~2000 particles, it had no such issue). I am just wondering what did I do wrong? Is it simply because I ran out of RAM?
> Best,
> Yangqi
>
> ########################################################################
>
> To unsubscribe from the CCPEM list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
########################################################################
To unsubscribe from the CCPEM list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCPEM&A=1
|