In case it’s useful to anyone else - couldn’t figure out why star_replace_UVA.com wasn’t working, so ended up writing my own scriptlet.

It only works for complete star files though - it assumes that the order of the lines in each _local.star is the same as the order in the particle stack, and the output needs a little massaging, but it works.

Cheers
Oli

#!/bin/bash
#run as ./cat_local.com *local.star
#you’ll need to add the header back after (with an additional field label for the image name
rm all_local.star #remove output if it already exists
touch all_local.star #make a fresh star file
for i in "$@“ #for each local.star
do
tail -n +19 ${i} > local_tmp.star #grab everything but the header from local.star
awk '{print $0, "\t", NR"@Particles_for_local/mics/"$1"s"}' local_tmp.star > local_tmp2.star #print each line to a temp file, and add a field corresponding to the image name
cat local_tmp2.star >> all_local.star #append the modified lines to the resulting star
rm local_tmp.star #get rid of temp files.
rm local_tmp2.star
done

> On Mar 2, 2017, at 5:32 PM, Oliver Clarke <[log in to unmask]> wrote:
>
> Eh, spoke too soon. It still isn’t substituting the local ctf values, just rearranging the lines. But at least it is not deleting the U/V/A columns entirely now, not entirely sure whether that is progress however.
>
> Cheers
> Oli
>> On Mar 2, 2017, at 5:21 PM, Oliver Clarke <[log in to unmask]> wrote:
>>
>> Fixed it! Edited lines 84-99 of star_replace_UVA.com and now it works for both test and my data. Basically altered the way variables were referred to. No idea why that fixed it, but there you go.
>>
>> Cheers
>> Oli
>>
>> Fixed:
>> gawk 'BEGIN{}/mrc/{\
>> if(FILENAME==ARGV[1]){image=$raw_rlnImageNameIndex; split(image, strimage, "@"); ImageId =strimage[1]; data_raw[$raw_rlnMicrographNameIndex, ImageId] = $0; } \
>> if(FILENAME==ARGV[2]){ImageId = $local_keywordNplus; data_rf[$local_rlnMicrographNameIndex, ImageId] = $0; } }\
>> END{\
>> for ( i in data_raw ) {\
>> n_raw=split(data_raw[i], str_raw, " ");\
>> n_rf=split(data_rf[i], str_rf, " ");\
>> for (j=1;j<=n_raw;j++){\
>> if ( j == $raw_rlnDefocusUIndex ) printf("%s ",str_rf[$local_rlnDefocusUIndex]);\
>> else if ( j == $raw_rlnDefocusVIndex ) printf("%s ",str_rf[$local_rlnDefocusVIndex]);\
>> else if ( j == $raw_rlnDefocusAngleIndex ) printf("%s ",str_rf[$local_rlnDefocusAngleIndex]);\
>> else printf("%s ",str_raw[j]);\
>> }\
>> printf("\n");\
>> }\
>> } ' $raw_starf $all_localstarf >> $new_starf
>>
>> Original:
>> gawk 'BEGIN{}/mrc/{\
>> if(FILENAME==ARGV[1]){image=$'$raw_rlnImageNameIndex'; split(image, strimage, "@"); ImageId =strimage[1]; data_raw[$'$raw_rlnMicrographNameIndex', ImageId] = $0; } \
>> if(FILENAME==ARGV[2]){ImageId = $'$local_keywordNplus'; data_rf[$'$local_rlnMicrographNameIndex', ImageId] = $0; } }\
>> END{\
>> for ( i in data_raw ) {\
>> n_raw=split(data_raw[i], str_raw, " ");\
>> n_rf=split(data_rf[i], str_rf, " ");\
>> for (j=1;j<=n_raw;j++){\
>> if ( j == '$raw_rlnDefocusUIndex' ) printf("%s ",str_rf['$local_rlnDefocusUIndex']);\
>> else if ( j == '$raw_rlnDefocusVIndex' ) printf("%s ",str_rf['$local_rlnDefocusVIndex']);\
>> else if ( j == '$raw_rlnDefocusAngleIndex' ) printf("%s ",str_rf['$local_rlnDefocusAngleIndex']);\
>> else printf("%s ",str_raw[j]);\
>> }\
>> printf("\n");\
>> }\
>> } ' $raw_starf $all_localstarf >> $new_starf
>>
>>
>>> On Mar 2, 2017, at 5:02 PM, Oliver Clarke <[log in to unmask]> wrote:
>>>
>>> To add another data point - when I run star_replace_UVA.com using the example Gctf_1.06 dynactin data, it also doesn’t work - gawk gives a syntax error (below). Could there be a version incompatibility? My gawk is 4.1.3, running on Ubuntu 16.04.
>>>
>>> Cheers
>>> Oli
>>>
>>> Error:
>>> user@narwhal:~/gct_local_test/Gctf_v1.06/examples_local_ctf$ ./star_replace_UVA.com micrographs_all_gctf.star Micrographs/*local.star
>>> original star file: micrographs_all_gctf.star
>>> gawk: cmd. line:2: if(FILENAME==ARGV[1]){image=$; split(image, strimage, "@"); ImageId =strimage[1]; data_raw[$1, ImageId] = $0; }
>>> gawk: cmd. line:2: ^ syntax error
>>> new star file micrographs_all_gctf_localUVA.star generated.
>>>
>>>> On Mar 2, 2017, at 10:37 AM, Oliver Clarke <[log in to unmask]> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I used Gctf 1.06 to run per-particle CTF correction, which seemed to work fine - it generates the appropriate _local.star files, and the output looks sensible. However, when I run star_replace_UVA.com to replace the global defocus values with local ones, it instead just deletes the defocus columns entirely in the output star. Has anyone run int this issue (and solved it)?
>>>>
>>>> I notice that the _local.star files have an additional column for phase shift which is not present in the example _local.star files, which were generated I presume with Gctf 0.50 - could this be the issue?
>>>>
>>>> Cheers
>>>> Oli
>>>
>>
>