You can't just respect the terminal colors because the colors you add wouldn't be readable.
Right now the best option is to define the skin in the config file (see https://dystroy.org/broot/documentation/configuration/#white...) and I've added a new issue requesting that broot detects if a white skin seems more suitable.