Yeah now I see it even has the phrase "core business", which he says is vague. I don't think it's that vague if you just add the qualification about "down the stack".
It's hard for me to think anything that should be brought in house that isn't down the stack, i.e. on the critical path to serving Twitter. It's just not obvious to outsiders what is down the stack for a given company (the "I could write Twitter in a weekend" people), but it should be more obvious to people working there.
e.g. the example of expense reports is obviously not down the stack -- if it goes down or is generally terrible, you can still serve your customers. As another example, in the old days, the big tech companies used to actually hire chefs and kitchen staff themselves. These days I believe it's all done through contractors.
Also I'd say the post assumes that saving money is always important... in the earlier stages of a company, they are frequently really wasteful and prefer to grow the business. An example is Dropbox starting on AWS and then building their own data centers to become cost efficient.