A couple of things:
(1) Perhaps the groups are not different.
(2) You might have outliers in your data - if you plot the values for each subject, do you see that a few subjects are not consistent with the rest of the subjects.
(3) Did you inspect the grey matter segments before creating the model? If not, I would look at the segments and make sure that the segmentation worked. Check for normalization, image artifacts, homogeneity of the tissue.
(4) Motion will be visible of the T1 as the image will not look clear, but blurry and with ringing lines in the brain.