Isn't the DK effect something that should be expected, given how the human mind works? To keep plowing ahead despite adversity, you need to have some trust in your abilities. It seems as if erring slightly in favor of over-estimating yourself might be beneficial for motivation - as long as the error remains small.
Some people focus a lot more to be good at the job/skill, incompetent ones focus more on promoting him or herself. Result is quite obvious in short-run.
Being competent on a skill or certain job does not really correlate well with being competent on knowing thyself, either. I guess a lot of people are trying to find out what they really want, thus during that pursue become more competent. Incompetent people know very well they want, that promotion, and willing to cheat in order to get it.
People both competent in skills and self knowledge tend to become more confident. That's the kind of confidence both solid and humbling. Not some empty bubble can't carry any weight that can only exist in big company environment.