How To Manage A Tech Career
antipatter.com
antipatter.com
On a related note, I often think about this quote from Benjamin Franklin:
"An investment in knowledge always pays the best interest."
Working in larger companies earlier on - when you're fresh out of college, for example - can often ensure you end up learning a very tiny portion of the development lifecycle. Moving from there to a smaller outfit, where you would assume greater responsibility, would leave you realizing that you know very little about the big picture.
Of course, much of that may be because the small companies were two startups that went nowhere, and the large company was Google. YMMV.
I think it really depends on what sort of company you work for. Small companies are highly variable, but there are many where the product is expected to scale from the start (because scalability is a functional feature of the product), have internalization from the start (because of the market it's aimed for), be secure and offer a latency SLA from the start etc. There are frequently open source projects that contain great deal of legacy code that have to be revived and modernized.
My previous could be described that way despite having been started by a few hackers. Of course they weren't in consumer web area: the product is an email/web security system: obviously it has to be secure, it can't be slow, it should deal with messages in different languages and it has to scale to email volumes of government organizations, universities and f1000 companies. Since we were in the email sphere, most of us or code base had been in C++ and Perl, which (by the virtue of the languages' age) had a great deal of legacy code out in the open that was nonetheless very useful.
It may be the case that these are after thoughts in small consumer Internet companies, but there are plenty of companies which employ similar skillset (especially if the application has a web UI and is deployed "in the cloud") who have to deal with these issues form the start.
I now work for a private consumer Internet company at this time where all of these issues also matter tremendously, but we are medium size at this point. Interestingly enough I've learned a lot more of "how to scale" here than I have at Yahoo: at first it was surprising, but realized it was almost tautological -- scale is about the first derivative, not the absolute site; going from 20,000 machines to 25,000 machines (not real numbers, just an example) is simpler than going from 1000 to 2000 machines. I can't just say "let's use this wonderful partitioned database and this wonderful data replication bus", I have to build those tools first :-)
A formula that works for me is: take an offer where you have the opportunity to learn the most. All else is fluff (they can talk about how great it is that they're small company, but they can still be structured in the same way that a big company is). It's also a good idea to work at large, small and medium companies in different industries to see what suits you better rather than make a decision based on what someone on the Internet (including some bozo calling himself after a libc function) wrote.
I've never worked on anything bigger than 10 machines or so, but is this really true? I would think the difficulty of scaling is not any kind of smooth function, but very easy or very hard depending on the bottlenecks you hit at any particular point.
lemma: doesn't every application have some theoretical limit about how big it can scale within the limitations of current network and computation hardware?
You are right it's not a smooth function. This isn't just about numerical aspect, it's figuring how to get over a hump: going from one datacenter to two is much harder (not even twice as hard) than adding a rack to two datacenters or adding a fourth datacenter when there are five.
In other words, when you're at scale you have tools available to operate at that scale. When you find yourself having to scale out without having done so in the past, you have to build those tools.
It's not clear to me that all of these are parallel goals. For example if all you care about it optimizing for salary then getting a job at a small obscure company is likely not the best way to go unless it's all you can swing. Likewise I can think of a few jobs that have a high "cool factor" but make it hard to break out into other areas if you ever want to change paths.
That is to say that depending on what you want you might need to go about it in a different way. The bit about learning new things though is solid advice.
For someone who may not be a political animal jumping into a new situation can be challenging. The benefit of working in the same place for a while is you get to know everyone there. Sometimes at a big company that means realizing that everyone there has rotten zombie brains, but there are usually some core of smart interested tech people who are influential.
I wonder whether most people would rate themselves above or below average in terms of risk tolerance. Is this something we tend to have a good picture of, or is it more like the Dunning-Kruger effect?
For the sake of argument, suppose that someone considers a 15% chance of going broke and having to live with their parents to be a "moderate" financial risk. Somebody else considers a 30% chance of the same thing to be a "small" financial risk. It's possible that the latter person has a higher risk tolerance. But it's also possible that they just like their parents a lot and wouldn't mind living with them as much as the first person.
My instinctive answer would be that most people consider themselves to be cautious or perhaps even cowardly - because, by nature, you pay the most attention to the risks that bother you the most, and if your internal, subjective mental process is to always avoid those risks, you will feel like a cautious person, even if there are other objectively-measurable risks that you are not avoiding.