My cod-biological explanation is that a child's brain is still forming connections and the process of becoming an adult involves pruning many of these connections to become more focused and efficient.
It's a fun little memory trick, and it was surprising to learn that it's not a universal experience. Not just "your letter-color association is different from mine" (it seems to be highly personal) but "wait, you don't do that at all?"
You might be associating orange with Friday because it makes you think of the sun and going outdoors?
It's all very interesting. Even more so because I don't think of days/colours. My mind gives the days size and mass. ie. starting with Sunday (I'm American) the day is small. And as the week progresses the days get larger, heavier and denser. Until Saturday, which is a big fat puffy ball of "day". You can do what you want with it and it will just be there and be your friend. I love Saturdays.
Nature abhors a vacuum, but it loves a rainbow.