To add to Marko's remark: I believe the main reason for this is the COST function. The affine registration in SPM uses a sum-of-squares COST function, which is less robust than the normalized mutual information (NMI) cost function of the rigid body registration in SPM. 

And: generally it helps to register multiple times, starting with lower degrees of freedom/parameters (e.g. co-registration includes only translations and rotations) before going to more degrees of freedom/parameters (affine = rigid-body + stretching and shearing).

Cheers, Henk