It’s fine as an occasional stylistic choice, but using it repeatedly as a regular synonym for brown is a pragmatic and collocational error. The meaning is clear, but the wording is marked, and overuse makes the speech sound odd in everyday contexts.