Why Invest in Tools?
medium.com
medium.com
Rapidly growing companies tend to over-hire, and then have bored engineers that spend lots of time creating wheel-building framework frameworks instead of working on wheels. Sometimes good stuff comes out of that, but there's a ton of waste, too -- in the worst case, you end up in a company that has hundreds or thousands of people, but only a core team of a few dozen are doing all of the actual work. So you have to ask yourself: would it just be better not to hire hundreds of engineers, and cut away the complexity and inefficiency that comes with an organization of that size?
I've seen it first-hand, and I've also seen friends go to big, famous unicorn companies and work on stuff that is far separated from the company's core business. It's sad, not invigorating.
This answers the question from a management perspective. Facebook has a lot to gain from a small number of huge success stories like React. This makes up for the cost of all the failed little tooling projects.
Your second paragraph is an assertion -- maybe it's true that one project like React makes up for all of the failed initiatives that the author describes (i.e. multiple person-years of effort), but that's a claim that deserves some evidence. Even if it's true, there's a level of management implied, or else everyone would spend their work hours chasing the same, captivating rabbits. My suspicion is that there's a lot more selection happening inside Facebook than the author lets on.
From experience, it's easy to get to a place where you have a lot of smart, well-intended people working on things that just don't matter. Saying "the best projects win" doesn't really solve anything, because you still have to define "best". Either that, or it's true that this is just a "champagne problem", and the Facebooks and Googles of the world can just throw dollars at anything, in the hope that something sticks.
Stellar games simply can't exist without stellar tools, and since every game is a little different, everyone's tooling needs are a little different. So you have no choice but to build your own tools (to some extent). Unlike armchair architects designing wheel frameworks or factory factories, the guy building the level design tool has real users (in this example: level designers) he has to build for.
This same model can be applied to web or service development. You just need to involve users from the start, especially so if your users are themselves developers who are especially demanding and critical and whose tools are especially difficult to create.
That's one approach, but it's not always the right one. It's only really necessary if your studio's selling point is pushing the limits, tech-wise. If your focus is the gameplay mechanics and engine and art are secondary to that, then focusing on tooling is more of a distraction than anything else. Hence why Unity and the now-quite-cheap UE are sane choices, since they let people focus on the gameplay.
General purpose software has the same pitfall that games do: people see the big studios/development companies spending a lot of time on tool-building, and they assume that's a prerequisite, which changes the mental model from "we need to start building the next cool _____" to "we need to build the tool(s) that will let us build the next cool _____". And that's a much more expensive model.
I think it's easy to get sucked into that trap for another reason too: developers know what developers want more than developers know what users want. That's why I agree with the last paragraph you wrote, that user involvement is key. I just think that companies need to be careful that they're not simply redefining users from "users of our core product" to "developers of the tools that will let us build our core product", since there's going to be some natural internal bias that way.
If you want to build something of a high quality in Unity you need structure and rigidity, and you need level designers and artists to follow guidelines and processes. So you still need tools and tools programmers. You might not have an entire team building an editor, but you need something (and it's very easy to build custom editor tools inside the Unity editor).
No disagreement there. I just meant that you don't need to start from a blank canvas and start in on engine, editor, etc. development just to get to the point where you can do good game mechanics. Obviously even if you use a canned engine you'll still need to do some work to get your asset pipeline squared away (just as how everyone needs to do a bit of scripting/config, etc. specific to their build process for example.)
I guess I didn't explain it well, but I was trying to contrast to the web world where people [seem to] very often look at places like Facebook rolling their own frameworks and saying "we need to do that to be good!" and thus literally go on to start from scratch.
If you have complex machines that are critical to the business and that take 6+ months before they are running at a reasonable capacity then it often makes perfect sense to have extra "dormant" capacity.
Probably not to the point where 90% of your capacity is "dormant" but IMO that is rare. A much bigger issue is all the companies running at over-capacity who are harming their long term success.
This paragraph is just a criticism of R&D, and when read that way, I think the counterarguments become obvious. Can large and/or rapidly growing technology companies (particularly Internet companies) afford to not invest heavily in R&D?
"Our job is not to just build Facebook, our job is to make the world more open and connected — and we in Product Infrastructure are tasked with giving the whole software industry the tools to help us accomplish this mission."
Then open source your social network and let it be hosted in a distributed manner around the world. Instead of shutting off access to APIs like this: http://techcrunch.com/2015/04/28/facebook-api-shut-down/
When I see companies like Twitter and Facebook become open source like Wordpress for blogs, then we can say they really care more about making the world more open and connected instead of building their own silo.
they want to open source their TOOLS and maybe even their model, but their data(ie their users)? gods no, that's their profit center, giving that up would be committing corporate suicide. Even if the engineers were willing & able to do that, the managers would kill it like a 1st trimester abortion.
Even shifting a portion to it might send a hell of a shockwave through the industry.
There's tons of potential in companies for IT innovation, whether small- or large-scale. It's proven by those, even tiny firms, that act on it. Most companies act against it on top of under-funding, under-staffing, and over-stretching their IT staff. That combination causes the effect you describe. It's not inherent in any way. If it was, examples I described wouldn't exist.
It takes a lot of work, and it feels quite different than the luxury of overt R&D and an emphasis on creating tools. We still mostly have to keep the development of tools to an as-needed basis and just take those opportunities to put an eye on the future. It's a delicate balance.
So to me the GP is not wrong when saying that these companies like Facebook have a serious luxury. That's the luxury to repeatedly experiment and fail with little relative consequence to the business. The rest of us can't afford to spend nearly as much time or make nearly as many mistakes.
So, there's certainly a huge gap between most companies and Facebook. Yet, even allocating one person, day, 15% time, etc. to try or build some new things might have quite some results over thousands to hundreds of thousands of companies. I'm just arguing companies could be putting in significantly more effort even without a true R&D department or luxurious business model. Past that, mileage varies from company to company esp with their circumstances.
The thing is whether they realize it or not every company is now has to be a software company, if they aren't then a software company will put them out of business within 10 years.
If this is explicitly true, I've never heard of anything like it in a company that doesn't sell to engineering organizations, and it's pretty amazing.
What moral obligation are you talking about? There's no moral obligation for anyone using open source software to give anything back. Are you aware of how software licenses work?
Contrast this with "they have a moral obligation." To me that is a much stronger statement: "People will think you are behaving in an immoral way if you do not do this thing."
It's funny how adding the word "certain" to the phrase actually makes it a less certain, gentler way of putting things - quite the opposite of the literal meaning of the word.
Put another way, I could imagine a law being passed to enforce "a moral obligation". But it seems much less likely to have a law enforcing something described as "a certain moral obligation."
[1] I mean "weak" in the sense of not being a strong claim of obligation, not "weak" in the sense that the phrase is a poor way to say something.
The moral obligation is real, and I personally feel quite strongly about it and I know most of my coworkers feel the same. Morality is relevant because it's a human feeling. It's easy to forget that even big companies like Facebook make decisions by the hands of single individuals.
Facebook was built on the LAMP stack >10 years ago. Without open source, Facebook wouldn't exist and we all know that.
I personally find satisfaction in sharing as much of the stuff we build as possible as open source in the hopes that others will be able to build something great.
BS!
Or they mean "Connected to Facebook alone"?
One tangential point I often think about is the importance of investing time in learning tools as a developer. What should the breakdown be? Should I spend 20% of my time "sharpening the ax" and the other 80% working on projects? Someone else in this thread pointed out that companies face some of the same questions. How many resources should be allocated to things like improving the build system or developing custom tools? Where is the break even point where time invested in tools starts saving time for developers working on core products?
It's totally fair to say, "thanks rich pplz, wut do WE do?" Though there's a ton of "rich pplz" out there that don't let engineers experiment the way that Facebook does. Perhaps Google. But the vast majority of these don't, I don't think: