Hi James,
If all or nearly all subjects have lesions in certain voxels, as you can see, it will crash. At the moment there isn't an easy solution for this on what concerns the GLM. However, there are things that you can do mitigate the problem. For voxels that are classified as lesions in all or nearly all subjects, consider using using a global mask (option -m) so that these voxels aren't used in the analysis.
To be more specific, you have 4 EVs that will always be in the design. Thus, no more than N-4-1 (i.e., N-5) subjects can have a lesion in a particular voxel. So, you can use fslmaths to compute a sum of the masks for all subjects, then threshold it at N-5 (or maybe at a smaller value, say, N-10, to give a bit of margin and avoid degrees of freedom that are too low). The resulting mask you can merge into the overall mask that you'd supply to randomise (-m), so that no test is done for these voxels, which are completely removed from the analysis.
About being slow or needing more memory: it's a consequence of having lots of voxelwise EVs.
Hope this helps.
All the best,
Anderson