Illustrator :: Applying Color Groups To Grouped Art Gets Randomized

I have a collection of color groups, that I am applying to grouped objects, but every time I change out the color group, it randomizes the order which the colors are applied.
 
In the kind of bulk work I'm doing, its really turning into an hours burner.
  
The specific process I'm using (I've tried several different ways with the same result)

1) Draw my art
2) Group the art
3) Create a new color group with my art selected.
4) Double click the color group and recolor.
5) Save the swatches ----- so far so good
6) Double Click a different color group (or select from the Recolor panel) -- here the other color group (swatches) get applied, but every time I do it, the order of which color is applied to