Hi,
I've been a developer for a few decades, mostly in large companies (>50k employees). On the subject of promoting technical talent, I can offer some insight into how it happens and why. I studied Organization Behavior and Operations Management as I found the topics interesting. Hence, I've learned how companies interact with their employees and how some companies adjust.
First, this "issue". A person is very good at their job and withing the company - they should be promoted to recognize their continued contributions. Most companies have a Human Resources department that is designed for efficiency and 'legally provable fairness', so people are categorized into titles, grades, and rankings. Most technical career paths have a limit to their maximum grade - hence a maximum salary.
For example, some companies have grades GSR01 - GSR08 as non-managerial employees. Manager roles have ranks from GRS09 - GRS15. Above that is executive, ESR16 - ESR21. Each has a salary range which is determined by competitive salary surveys of other companies in similar industries.
The situation described above - a talented person needs to be compensated more - causes a conflict because these companies don't have a "career path" that includes non-managerial positions with higher pay (or career management requirements). A manager's work performance is determined by a number of tasks that are related to the performance of their employees. But such a list isn't available for a company without a "Technical Career Path". The result is that in order for a good employee to get a promotion, they are forced into a Managerial Career Path. A technical person is moved into a management role because the company has no other way to promote them or increase their salary.
Allow me a specific example: A person worked as a mechanic at a local muffler place. He was their best. They had 4 other guys and a front-office manager, plus an owner. When the manager quit, the top mechanic wanted they job because it paid more than the mechanic job. The owner was quite blunt: "You have years of experience working on cars and no experience running a store. I can't lose you as a mechanic and a I can't risk my only income source to a person with no experience. So I can't promote you." (Later, they negotiated a different pay scale to keep the mechanic working there. Most companies don't have that flexibility - all salaries are limited by HR rules.)
If the above example is applied to a large company - they can afford to train a manager 'on the job' - so they accept the risk and cost of doing so. And in most cases, the person is good enough at the job. However, everyone involved knows it is not a best case scenario. It only happens that way because the company doesn't have a better solution.
---
I mentioned the better solution above - the Technical Career Path. This is a structure within a company that allows the desired progression. Often it requires the employee to meet requirements beyond their day-to-day work. Some requirements may be; nationally published, provide support on other products within the company, present technical information within a larger group within the company, contribute regularly to external projects (open source), etc.
For this to happen, the company has to invest in an additional career path within their HR department and train upper and mid-level management on managing, mentoring, and guiding these employees. This is not a small undertaking. Most small to medium companies cannot afford this type of investment.
The alternative, and the result, is that technical people don't get promoted to management very often. The company recruits from outside so they don't have to bear the cost of training the new manager.
---
So, are we, technical people, 'stuck'? Is this something that can only be solved by the 'company'? No. It is something that can be addressed by each person who ranked "Promotes from Within" as high on their list. Being bluntly; for a person to get promoted, they have to be promotable. That is, they have to develop the skills the company is needing. Take a couple of night courses on Business Management, Leading Teams, and Public Speaking. Then ensure the company management knows you have these skills and want to put them to use. We technical people will happily spend a weekend building a network in our basement or learning the latest javascript library. But in order to get beyond that promotion boundary, a technical employee has to learn something other than the technical.
As a technical person, I've resigned from management jobs in order to go back to technical work. I love creating software more than I do leading people - even for less pay. When I had access to the Technical Career Path, I took it and I was happy.
thanks,
jon
'legally provable fairness' is a made-up term that is intended to relate how an action by a person is driven by a company policy. Most company policy is written with the intention of being used as a defense in a legal situation relating to a complaint against the company. For example, an improper termination lawsuit or an investigation by a government agency.