This is separate from the technical categorization of colors. Rather, how do you associate color families in your thoughts? Also, how good is your color vision, for comparison?
For me, I have excellent color vision;
- Reds, pinks, magentas, very red oranges
- Oranges, Yellows, Browns, very yellow greens
- Greens
- Most blues, purples, very blue greens
- Whites, off-whites, creams, light blues
- Black, grays, very dark blues, and other near-black colors that aren’t browns
I also have very good color vision.
For me it heavily depends on the context. For example I have spent some time working out a color palette for my wardrobe, so now the colors I use there are heavily associated for me.
Without any context, it’s similar to your grouping, except for the second group: oranges go nearly all to the reds, yellows go to the greens, browns go to black and greys.