The procedure of cutting out electron density, putting it into a large unit 
cell, and backtransforming to get structure factors can be tricky (as 
you've discovered), so we put some instructions on our webpage:

http://www-structmed.cimr.cam.ac.uk/phaser/density_as_model.html

The last time I tried that it worked, but let me know if you run into any 
problems with your case.

Randy Read

On Nov 4 2007, Jan Abendroth wrote:

>Hi all,
>thanks a lot for the various responses. When I tried to use a map as the
>serach model, I ran into various problems:
>again, the starting point is a weak, yet convincing molecular replacement
>solution in the hexagonal crystal form (1mol/asu) and no MR solution in P1
>(2mol/asu, 2-fold in SRF).
>
>a) using phaser and defining the search model though DM map of the MR
>solution in the hexagonal form: Phaser stops as two space groups were used,
>p1 for the data set and P6... for the map
>
>b)
>- fft to create map after MR and DM of hexagonal form (map in P6..., asu)
>- mapmask to cover MR solution (in P6..., asu)
>- mapcutting using map and mask from prev steps (P6.., asu)
>- sfall to generate FC, phiC in large P1 cell:
>  "fatal disagreement between input info and map header"
>
>c) same steps as in (b), however, using P6... and full unit cell
>- mapcutting: maprot dies with "ccpmapin - Mask section > lsec: recompile"
>
>d) same steps as in (b), however, using P1 throughout
>- sfall dies with: "Fatal disagreement between input info and map header"
>
>e) same steps as in (c), however, using P1 and full unit cell - should not
>be different from case (d)
>- mapcutting: maprot dies with "ccpmapin - Mask section > lsec: recompile"
>
>Any ideas? I btw. use the osx binaries from the ccp4 webpage.
>
>Thanks for any input!
>Cheers
>Jan
>
>
>
>On 11/2/07, Edward A. Berry <[log in to unmask]> wrote:
>>
>> One other idea idea:
>> 1. Solvent flattening on the hexagonal crystal
>> 2. use the flattening mask to cut out the density of one molecule,
>>     put in a large P1 cell for calculating structure factors
>> 3. Use the structure factors from the density of the hexagonal crystal
>>     to solve the triclinic crystal by molecular replacement.
>> 4. If 3 works, multicrystal averaging to improve both crystals
>>     til the map is traceable.
>>
>> Jan Abendroth wrote:
>> > Hi all, I have a tricky molecular replacement case. One protein in 
>> > two different crystal forms: hexagonal with 1 mol/asu, triclinic with 
>> > 2 mol/asu (based on packing and self rotation).
>> >
>> > No experimental phases are available this far, however, there is a 
>> > distant homology model. For the hexagonal crystals, phaser gives a 
>> > solution with really good scores (Z > 9, -LLG > 50) and a good 
>> > packing. While the correct solution is way down the list in the RF, 
>> > the TF can separate it from the bulk of bad solutions. Slight changes 
>> > in the model give the same solution. Maps are somehow ok, however, not 
>> > good enough to enable arpwarp to build the model. It does not totally 
>> > blow up either.
>> >
>> > For the triclinic crystal form with 2 molecules related by a two-fold
>> > which is not parallel to a crystal axis, phaser does not find a
>> > solution. Neither does molrep using the locked rotation function with
>> > the two-fold extracted by the SRF.
>> >
>> > As the homology between the data set should be higher than between 
>> > the model in the data sets and the search model, I tried a cross 
>> > rotation function between the two data sets. Strong peaks there should 
>> > give the relation between the orientation of the molecule in the 
>> > hexagonal crystal (that I believe I can find). With two rotations 
>> > known and one translation undefined, I'd be left with only one 
>> > translation that needs to be found. Then averaging within P1 or cross 
>> > crystal might improve the density...
>> >
>> > Almn appears to be the only program in ccp4 that can do a cross 
>> > rotation using Fs only, right?? I used the P1 data as hklin, the 
>> > hexagonal data as hklin2. Almn comes back with two strong peaks (see 
>> > below), however, now I am lost: - the first two peaks appear to be the 
>> > same - are the Euler angles the ones I could use in a peak list for 
>> > eg.
>> Phaser?
>> > - does this procedure make sense at all?
>> > - any other ideas?
>> >
>> > Thanks a lot
>> > Jan
>> >
>> > almn.log:
>> > ##########
>> >   Peaks must be greater than     2.00  times RMS density     52.2161
>> >
>> >
>> >
>> >                         Eulerian angles                       Polar
>> angles
>> >
>> >                      Alpha   Beta  Gamma       Peak       Omega    Phi
>> > Kappa           Direction cosines
>> > PkNo Symm: 1   2
>> >
>> >  Peak   1
>> >       1   1   1      323.7  143.4   18.5      540.8        92.9   62.6
>> > 143.8        0.4594  0.8867 -0.0511
>> >       1   1   2      323.7  143.4   78.5      540.8        83.2   32.6
>> > 145.9        0.8364  0.5351  0.1184
>> >       1   1   3      323.7  143.4  138.5      540.8        75.6    2.6
>> > 157.2        0.9674  0.0441  0.2495
>> >       1   1   4      323.7  143.4  198.5      540.8        71.9  332.6
>> > 174.4        0.8439 -0.4373  0.3108
>> >       1   1   5      323.7  143.4  258.5      540.8       107.2  122.6
>> > 167.0       -0.5149  0.8049 -0.2950
>> >       1   1   6      323.7  143.4  318.5      540.8       101.7   92.6
>> > 151.7       -0.0446  0.9781 -0.2034
>> >       1   1   7      143.7   36.6   41.5      540.8       161.7  321.1
>> > 175.0        0.2448 -0.1974 -0.9493
>> >       1   1   8      143.7   36.6  341.5      540.8        20.4  171.1
>> > 128.2       -0.3451  0.0540  0.9370
>> >       1   1   9      143.7   36.6  281.5      540.8        31.6  201.1
>> > 73.8       -0.4882 -0.1885  0.8521
>> >       1   1  10      143.7   36.6  221.5      540.8        82.2  231.1
>> > 37.0       -0.6220 -0.7711  0.1363
>> >       1   1  11      143.7   36.6  161.5      540.8       144.3  261.1
>> > 65.1       -0.0902 -0.5770 -0.8118
>> >       1   1  12      143.7   36.6  101.5      540.8       158.6  291.1
>> > 118.5        0.1317 -0.3411 -0.9307
>> >
>> >  Peak   2
>> >       2   1   1      143.7   36.6   41.5      540.8       161.7  321.1
>> > 175.0        0.2448 -0.1974 -0.9493
>> >       2   1   2      143.7   36.6  101.5      540.8       158.6  291.1
>> > 118.5        0.1317 -0.3411 -0.9307
>> >       2   1   3      143.7   36.6  161.5      540.8       144.3  261.1
>> > 65.1       -0.0902 -0.5770 -0.8118
>> >       2   1   4      143.7   36.6  221.5      540.8        82.2  231.1
>> > 37.0       -0.6220 -0.7711  0.1363
>> >       2   1   5      143.7   36.6  281.5      540.8        31.6  201.1
>> > 73.8       -0.4882 -0.1885  0.8521
>> >       2   1   6      143.7   36.6  341.5      540.8        20.4  171.1
>> > 128.2       -0.3451  0.0540  0.9370
>> >       2   1   7      323.7  143.4   18.5      540.8        92.9   62.6
>> > 143.8        0.4594  0.8867 -0.0511
>> >       2   1   8      323.7  143.4  318.5      540.8       101.7   92.6
>> > 151.7       -0.0446  0.9781 -0.2034
>> >       2   1   9      323.7  143.4  258.5      540.8       107.2  122.6
>> > 167.0       -0.5149  0.8049 -0.2950
>> >       2   1  10      323.7  143.4  198.5      540.8        71.9  332.6
>> > 174.4        0.8439 -0.4373  0.3108
>> >       2   1  11      323.7  143.4  138.5      540.8        75.6    2.6
>> > 157.2        0.9674  0.0441  0.2495
>> >       2   1  12      323.7  143.4   78.5      540.8        83.2   32.6
>> > 145.9        0.8364  0.5351  0.1184
>> >
>> >  Peak   3
>> >       3   1   1      335.2   54.5   36.5      209.2        78.8   59.3
>> > 55.6        0.5006  0.8437  0.1940 ...
>> >  Peak   4
>> >       4   1   1      155.2  125.5   23.5      209.2        62.8  155.8
>> > 179.4       -0.8112  0.3638  0.4579 ...
>> >  Peak   5
>> >       5   1   1      349.3   53.8   13.0      176.4        87.7   78.2
>> > 53.9        0.2051  0.9779  0.0406 ...
>>
>