In my experience, if you love technology and tinkering, you should absolutely not work at a startup.
In the first years, a startup's main objective is results, results, results. Now is not the time to play with the latest node.js module, it's about picking the technology that will allow you to reach your goal the fastest, period. Compromise on that and another startup built on PHP will beat you to the punch.
If you love tinkering, join a big company. Spend a few months there to prove yourself to your colleagues and superiors and once you have that, you will have no problems carving out a comfortable 20% time that only you know about that you can use to tinker with the latest hotness.
Just don't do that at a startup, or be prepared to be the reason why the startup failed.
However, to me love of technology is not playing with the latest node.js modules. While node.js is not my cup of tea, I can see how that be fun (I enjoy playing with the latest OCaml libraries, C++11 features, golang, etc..), but ultimately that alone isn't about technology. To me building technology is about reading a paper, extracting the core ideas, and applying them knowledge to implement a feature customers and no competitors offer is.
If we take this definition of technology, then there's plenty of place in startups for people who love technology. It depends on what the startup is building: if innovative technology (as opposed to an innovative business model) is at the core of the company's product, it's a great place for technologists. (There is of course nothing wrong with innovative business models -- nor are innovative business models mutually exclusive with innovative technologies; indeed, some of the greatest companies have combined the two!)
Now with big companies it is absolutely true that even if technology is not the core, challenges around availability, scalability, performance, etc... necessitate more than a trivial investment in technology.
"Startup" and "big company" are also not binary. I've joined LinkedIn when it was may be 200-300 people and even then we were solving some rather tough technical problems (you can read about some of the problems we solved in this paper that was accepted to USENIX FAST -- http://static.usenix.org/events/fast/tech/full_papers/Sumbal...). Although, the focus would have been more immediate had I joined LinkedIn when it was a 10 person startup -- but even at fairly early stages, the team was solving problems related to search and graph traversal, etc...)
There are people writing line of business code in 5000 person companies and there are people implementing query optimizers in FPGA in 10 person startups. It's very hard to make any generalizations and make categerical statements such as "if you love technology and tinkering, you should absolutely not work at a startup" or "to enjoy startups you have to love technology" (there are plenty of great startups founded by people who loved something more than they loved technology -- and these startups have made my life much more enjoyable as well!) I've done work that I am proud of in companies ranging in size from 15 to 14,000 people and learned tons in all of these environments (with various trade offs for companies of different sizes).
(Shameless plug: I hack on distributed systems in C++ at Cloudera -- and we do plenty of other cool things, despite not being a big company. Like distributed systems, databases/data stores, file systems, query processing, and more? Drop me a line.)
There are some exceptions for startups whose real pitch is precisely the R&D, and who are able to pull large amounts of up-front funding from investors who are willing to bet on the technological development. Tesla is one example, in part because Musk's large amount of self-funding meant it could afford to take a full 5 years between founding and product-to-market. Very few startups have 5-year R&D runways, though; if you want to do R&D on that timescale you're usually better off at a place like Google or MSR, in parts of academia, or even a big engineering company (BBN, Lockheed, Intel, etc.).
Startups are the best hope for R&D to make it out of the lab. You have been brainwashed by the Lean Startup movement. It's a good strategy but it is not the only way to skin the cat, and if you are looking to change the world it's probably a bad way to do it.
Look at the greatest companies that have come out of Silicon Valley over the last several decades, and you will see companies which started out with products that took a lot of time, capital, and R&D to launch. The two largest market cap companies in the world, Google and Apple, were not created through the Lean Startup approach of throwing together existing technology and pivoting around trying to find a local optimum, but came about from the hope that if you spend your time up front, get investment, do the R&D and hard work, and build something with an eye towards the future, you will be rewarded. Intel. HP. Microsoft. Should I keep listing companies that did big R&D up front?
Who do you think is more likely to change the world, the latest startup that throws together some B2B app on AWS with no vision and just is trying to find product market fit, get their Google acquisition, and live happily ever after, or companies like Oculus, Meta, MakerBot, hell even Soylent, who have put in serious big-bet, R&D, and say lets shoot for the moon or lose everything trying? I'm not saying that the lean startup approach shouldn't be done ever, but to get up and claim that startups are not the place for R&D basically goes against everything that makes Silicon Valley an amazing place. If this is the mentality that comes out of the pop-culture-ization of startup life I don't want to be a part of it.
I think startups can indeed be a good way to take existing R&D and productize it, i.e. "make it out of the lab", but I don't think they develop a lot of it.
For Microsoft the reference there was not their (extremely lucky) deal with MS-DOS but all of the other work they did: from the early BASIC days to the days of Windows where they did extensive R&D in order to leapfrog their competitors.
You ignored my other examples but the fact is that the Lean Startup approach to startups is a relatively new thing, only really fits for web-oriented startups where up front development costs are low thanks to SAAS and open source, and even then the companies you might be able to point to as definitive success stories are YC companies and maybe companies that are the latest round of tech IPOs taking a Lean slant on how they found their product market fit. Eric Reis's startup failed! Find me a biotech startup, find me an energy startup, find me a robotics or hardware startup that doesn't do R&D on day zero. The YC model of developing a web application using open source software and launching on Heroku to get revenue for a B2B need is a very narrow slice both in time and space of the way innovation happens. Hell, look at any legendarily successful software companies that aren't in the web sphere of the last 10 years, like iD Software, Adobe, Pixar, Wolfram, and so on and you see R&D focused companies from their beginning.
The approach of doing a university spinoff using your own tech is a model I do like, and I agree it's a way to develop new tech and also bring it out of the lab. But I think the academic part there is key: you get some R&D runway up front before you start the startup and need to make money, rather than doing a startup up-front. You can then "afford" to start the startup once at least some of the high-risk work has been done and you're reasonably close to a product. Since you mention Wolfram, that's probably an even better example of that than Google, given Wolfram's rather lengthy prologue to his startup: he spent 8 years at Caltech (1979-1987, first as PhD student, then as professor) developing the Symbolic Manipulation Program, a precursor to Mathematica. Then he spun off a company in 1987 and launched Mathematica in 1988. I don't think he would've been able to do the same kind of initial R&D in a startup as he was able to do at Caltech.
I'd think you'd want most or all of the experimenting to be out of the way.
Facebook started on PHP and Google on C++.
The software innovations that these companies brought to the world didn't happen in the start up phase but years later, once they had grown sufficiently big that they had resources to devote to R&D in an effort to streamline and optimize their existing infrastructure.
I talk to a lot of startups on a daily basis, most of them are either built on Java or Ruby on Rails. I don't see any of them building their v1 with Clojure, Kotlin, Ceylon, Idris or Ermine or any technology created in the past five years.
tldr; hipsters will grossly overestimate their ability to "change the world" with that "brilliant idea" they had this morning in the shower, so starting a new company is pretty much the worst path to take, if you want to "tinker with technology".
You've listed people whose stories bear little resemblance to the vast majority of little tech startups that subsist on ramen and go through accelerators.
- Google: C++
- Microsoft: assembly
- Apple: assembly
- eBay/PayPal: Java/C++
Like I said, there is simply no room in the early years of a start up to experiment with recent technology. I'd go further and claim that it's actually critical to use older, established technology to get your startup off the ground so you can focus on your idea without being hampered by technological glitches.
Viaweb: lisp. ITA software: lisp.
It does mean that you need to be pragmatic and prepared to pick the sub-optimal choices and check your ego at the door for things to work well. You can love technology for what it lets you achieve without getting hung up in not having the ability to always do things perfectly.
If you can deal with that, chances are that if you pick a startup that is technically challenging you will also get to tinker. The constraints are just different.
I've certainly had people come into dev teams in startups and get annoyed at technology choices and constantly argue for changing them (one even tried to get me fired, and almost ended up getting fired himself as a result because the way he went about it backfired spectacularly) because they disagreed with choices that laid firm not because they necessarily were ideal any more - startups change quickly - but because they were good enough and taking the time or resources for a change would have been totally unacceptable.
If you're that guy who needs things to be perfect and who will put in lots of time and energy trying to change things that aren't perfect, then a startup is not right for you - you might get lucky and agree with everything, but more likely you'll find yourself endlessly frustrated because people refuse to listen to your suggestions to rewrite stuff unless they are necessary right now.
But even though I insist on the pragmatic choices in a startup, I've also had plenty of opportunities to tinker at startups. The issue is that you need to constantly think about what gets the job done most efficiently and fast, without too much technical debt, not what is ideal.
You also need to be an earlier employee to get that freedom at most startups - if you get in as one of the first few tech hires you have lots of freedom to make architectural decisions and tinker (as long as you're focused on the company goals. But then there certainly is a long period of slogging through things that might not be very exciting before the company (hopefully) gets large enough to be able to afford more unfocused/unstructured tinkering on the side as exploration again.
This need not be essentially true. Although delivering results is of highest priority, delivering "right" results is even more important, results that create your USP, results that set you way ahead of your competitors. And that requires right amount tinkering, if you want to call it that. And that tinkering is a lot of fun - to deliver world class results in the least amount of time possible.