Comment by indymike | Hacker News Reader

Comment by indymike | Hacker News Reader

Parent Full thread

It's not generalist vs. specialist, it's capacity for learning and adaptability that really matters.

When things are steady state, learning capacity and adaptability don't really matter. You do the same thing, the same way, every day. When change is in play, and it doesn't matter if that is technical or organizational, those who can learn faster and change behaviors to meet the change are going to outperform.

I once was given a team of interns and newly minted developers. In 90 days, we were able to write an entire new ETL system while the legacy team struggled to just keep their steady-eddie system running. We replaced 10 years of code and a kill and fill system with a fully differential pipeline that reduced load on the upstream system by 98%. Literally from millions of documents to process every night to a few thousand. Why? Our team was hired for capacity to learn and ability to adapt. The old guard was hired for experience and knowledge. Yes, they knew their stuff. The problem was they were not able to deal with change at all. The best part? Most of the team had never even heard of ETL and knew zero SQL.

Was the legacy team free from daily responsibilities of maintaining the system or were you at an unfair advantage because you only had to work on new development?

This is probably the only way that anything gets done though. Someone has to keep the legacy system running, warts and all, while someone else has the free space to work on something better. The only thing that's misguided is comparing the two as if they were tasked with the same goal (they were not).

But, one can speculate that if the situation was reversed, and the interns and newbies were forced to maintain the old system and the old hands were given carte blanche to write a new system, that maybe they wouldn't have been as creative or open-minded about it. Or maybe they'd have deeper insights based on their experience? Who knows?

Richard Hamming talks of the development of FORTRAN and Algol in his History of Computers talk. He recalls FORTRAN being received as something between an abomination and a stupid trick.

Algol, in contrast, was backed by many international groups and is described as "an attempt by the theoreticians to greatly improve FORTRAN" (by way of all their great, formalized knowledge and 'wisdom').

The seasoned experts always think they have some secret that the upstarts are missing, a special sauce that no "true solution" built by "true engineers" would be without. And then their overengineered designed by committee and compromise solutions stumble out of the garage and collapse in a heap. Meanwhile, those things built by people more motivated by a desire and a passion than by signaling their expertise soar past and into fields and uses their designers never imagined.

People who have been in the trenches for years tend to have difficulty ignoring the current system and its limitations when building a replacement. So they end up rebuilding the same product over again.

I've seen it a lot and have been on both sides. Newly created teams have a massive advantage when it comes to replacing existing systems because they avoid getting bogged down in the details. However, they do invariably miss implementing key features in the new system, leading to a situation where the new system is better than the old one only 80-90% of the time.

> Richard Hamming talks of the development of FORTRAN and Algol in his History of Computers talk. He recalls FORTRAN being received as something between an abomination and a stupid trick.

Interesting. I recently designed an executable format that runs on seven operating systems and I've definitely been getting that vibe from a lot of my colleagues. I try not to take it personally since I feel like some ideas can only be discovered and the will of technology often isn't our own.

We were the maintenance team for the old system. We were allowed to only work on the parts of the etl pipeline that required updates due to changes in the data we were extracting. The more experienced team members were dealing with mostly database issues caused by trying to load 8 million documents every night.

> It's not generalist vs. specialist, it's capacity for learning and adaptability that really matters.

From a career perspective, you are going to get a lot less opportunities to show off your adaptability without specific (specialized) experience, today, that impacts your career. You need to be in the right place to make an impact, have someone who notices your impact, have a space for you to fill to move up in the hierarchy. These things rarely happen. This is the root of the answer to the question. Specialization wins.

The focus on skills and knowledge is a gross mischaracterization of the problemspace, outside of the article's assumptions and the soft terms "specialization & generalist". It's hard to argue that attractive sociopaths with political connections/wealth (which is a very specialized niche) do the best in American culture by most metrics. Specialization wins.

Reply on news.ycombinator.com