If those words and others were more obscure I’d agree with your point, but they really are common enough that I think Aoi can mean blue or green in nodern Japanese.
But when ao is used outside of a set phrase, to tell somebody what color you're talking about, it means blue.