I'm sure you didn't intend any offense, but for future reference, "colored" is considered archaic and borderline racial-epithet in the U.S. (I realize it's quite normal and non-offensive elsewhere.)
No one blames you for not knowing; all you're responsible for is how you respond and correcting future usage. The appropriate response is something along the lines of, "I didn't know. I will say X in the future. Thanks for the heads up." And, if you haven't been told what a better option is, you can generally ask without causing upset.
But no worries, honestly, except for this minor slip, you seem to be doing well.