Dear all,
I try to impute a SNP-dataset (ca. 600.000 SNPs) using a 1000G-haplotype-reference-panel with impute2 version 2.3.0.
I am working on two clusters with 64 processors each. The operating system is Linux version 2.6.32-5-amd64 (Debian 2.6.32-48squeeze4)
I ran into the following problem with impute2, which I could not solve up to now:
After cutting the data into 2mbp chunks I need to run ca. 1400 processes in parallel.
After starting e.g. 60 processes, everything seems to work well for a while. However, after maybe 1-3hours, the processes show a strange behavior. Some just go through as they should, but most of them get stuck then using on average less than 10% CPU. These processes keep on running, and although they use almost no CPU, they seem to slow down the computer.
Impute2 does not produce any error messages or log-files.
I already tried the following:
1. I called the different jobs with a pause of 5s between two consecutive calls.
2. I tried to call impute2 together taskset, a linux-command distributing jobs to specific cpus. (as I guessed it could be a problem due to the assignment to CPU's)
3. I use impute2 together with Linux's nice-command (nice -n19). Omitting it, did not change anything.
4. I tried to tune the parameters of impute2 (First, I used tried imputation without pre-phasing. Now, I use imputation with pre-phasing. The problem persists in both cases.)
After the end of my message is the part of the script showing the options I used for imputation with pre-phasing.
Does anybody know this problem or have any idea, what it could be?
Many thanks in advance for any help and best regards
Birgit
##############
#Pre-phasing
##############
nice -n19 \
impute2 \
-prephase_g \
-m ${Reference_path}/genetic_map_chr${i}_combined_b37.txt \
-g ${Input_path}/Chr${i}_b37.gen \
-strand_g ${Strand_path}/Chr${i}_New.strand \
-Ne 20000 \
-buffer 250 \
-k 80 \
-iter 30 \
-burnin 10 \
-include_buffer_in_output \
-o ${Outdir}/Chr${i}.pos${low}-${high}_g37.gen.phasing \
-int ${low} ${high} > ${Outdir}/Chr${i}.pos${low}-${high}_phasing.tmp
sleep 5
#################
##Imputation
#################
nice -n19 \
impute2 \
-use_prephased_g \
-known_haps_g ${Outdir}/Chr${i}.pos${low}-${high}_g37.gen.phasing_haps \
-h ${Reference_path}/ALL_1000G_phase1integrated_v3_chr${i}_impute.hap.gz \
-l ${Reference_path}/ALL_1000G_phase1integrated_v3_chr${i}_impute.legend.gz \
-m ${Reference_path}/genetic_map_chr${i}_combined_b37.txt \
-Ne 20000 \
-buffer 250 \
-k 80 \
-pgs_miss \
-phase \
-o_gz \
-o ${Outdir}/Chr${i}.pos${low}-${high}_g37.gen.imputed \
-int ${low} ${high} > ${Outdir}/Chr${i}.pos${low}-${high}_imputing.tmp
sleep 5
To unsubscribe from the list visit this webpage https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=OXSTATGEN&A=1
