Building Building Blocks
segment.com
segment.com
I think Metalsmith and Khaos are pretty good examples of why we feel the need to build our own.
The crux of it is that the alternatives (Yeoman and Jekyll) aren't designed to be augmented without opinion. They are designed as "solves all your needs tools and gives you the preferred workflow".
Metalsmith doesn't claim to solve all your needs. It actually doesn't solve any of your needs out of the box, unless you need a really complex way to copy files from a directory to another without doing anything else to them. It defines an extremely simple core—an API that extendability can be based around—and then hands over all of the logic to plugins. Which means it's pretty much never going to get in your way.
Similarly with Khaos—which we haven't really perfected yet—we're looking for an extremely simple core API design that people can augment to solve their scaffolding needs. We want it ourselves to scaffold integrations for Segment, to scaffold new Node and Go projects, we're even thinking about scaffolding Segment-branded PDFs.
Yeoman doesn't think that way, evidenced by this paragraph from their landing page:
> Through our official Generators, we promote the "Yeoman workflow". This workflow is a robust and opinionated client-side stack, comprising tools and frameworks that can help developers quickly build beautiful web applications. We take care of providing everything needed to get started without any of the normal headaches associated with a manual setup.
Khaos doesn't prescribe any workflows or client-side stacks, it just lets you scaffold things. What you want to scaffold is up to you, as it should be. You can use it for eBooks for all we care :)
Khaos also happens to be built on Metalsmith. It's just a series of plugins that run before the files are written to an output directory. That wouldn't have been possible with Jekyll, we'd have had to roll our own again inside Khaos or do some incredibly hacky shit to get it to work.
---
All that said, some of our projects suffer from NIH. And when they do it's usually clear over time since we get sick of dealing with them and we use something that has more widespread support behind it. But for others, we're super glad we've rolled our own because we get tired of other people's opinionated software :)
There are definitely lots of existing pieces of software that we like to use though. I've been thinking recently it would be cool for companies to have a Chrome Extension that the engineering team could use that specifically marked GitHub repositories as "liked/disliked" in terms of adhering to the companies design ethos. That way we could keep track a lot easier of the stuff we like to default to, so that we don't have to roll our own all the time.
It did get annoying to deal with the mechanics of lots of repos. So we built tooling to make that easier. For example, CLI commands like "goto analytics.js" will clone and take us to the local copy of the repo. And "publish patch" handles all the mechanics of updating History.md from the git log, incrementing the version appropriately in package.json and component.json, tagging the commit and releasing to github and npm. Khaos, also mentioned in Sperandio's article, helps us template out new repos quickly. With a few pieces of tooling like that you can move pretty fast across lots of repos.
There's also an interesting balance for things that technically should be their own repository, but shouldn't for workflow reasons since it actually means they will be more quality if they are grouped and updated at the same time. Where splitting things out can actually discourage updating if the cost is too high.
There's also a big open question around what should be a separate repository, and what is separation just for separation's sake? A similar kind of question that has to be answered for individual functions in a codebase. We've made the wrong call sometimes and split things out that should have really been together.
I tend to think that the drawbacks are worth it, since it means things are actually decoupled and generally more parts of the codebase are able to get to a "finished" state. Lots of the modules we've built haven't been touched in months or years now since they do exactly what they need to do without any bugs now.
In general I think that in programming we talk a lot about building things that are decoupled and simple, but we haven't invested anywhere near enough in tooling to quell the drawbacks to things being more spread out. The pros are extremely valuable, and the cons are solvable, we just haven't done anything about it.
Rather than go to highlightjs's website to download a custom build of the script that doesn't have 22 languages (20 of which you probably don't need), adding it to your asset pipeline, then repeating whenever you need to add a new one, with highlight's plugin approach + a simple package manager and build process like duo, you can easily add and remove each language from your build ad hoc.