Hacker News Reader

Comment by zug_zug | Hacker News Reader

I went from 10 years in startup to fang and felt fang was awful.

Everything was built in-house, and none of it held a candle to industry-standard tools. Every single thing: observability, containerization, CI/CD, code-review, source-control, etc was mostly written in PHP with seemingly no UI input and no documentation.

The economy of scale largely favors the startup, because they simply rent best-in-world tools hosted by another company that has hundreds of people dedicated to source control or IDE (or whatever) versus 20 engineers a FANG company would put on it.

View on HN

Those tools are affordable until the company making the tool needs to start actually making a profit and they raise their prices or they decrease their LOS. Then the tools become an expensive risk and the price increases end up percolating into your own startup. If you raise prices to compensate, you'll affect your growth. That's how startups go under.

In a ZIRP world using the tools of other startups makes sense but the stability of the FAANGs and their in-house tooling means they can weather much choppier economic conditions than the equivalent startup.

I guess that's why so many SaaS startups have a policy of "you have to pay for our software but we never pay for software".

Link some of these mythical industry standard tools for the rest of us, please. And share how you’re operating them with a small startup that they’re working so well.

Absolutely

AWS: Superior to the cloud compute at the Fang I was at. Best-in-class API, and documentation, and support on-call. Tenancy issues are rare.

Docker: Run any application on any server with one command. Isolates all state from that application to one-directory. Allows spinning up test-environments with your whole stack (~6 applications or so if you're a small team, including DB) on a server in a few minutes with docker-compose.

Github: Solid UI design, lots of little niceties, adequate search.

Splunk: Best-in-class logging solution (the place I was at asked you right-click download 200megs of logs and grep them). Can be intergated with Jenkins

Jenkins: Can do almost anything well, easily with the right plugins. Acts as both CI/CD job system and scheduled job system with full premissioning, history for auditing, logs, supports dynamic workers easily with AWS.

Packer, artifactory, slack for integrations, Jira (which can represent complex workflows well)

------

How they work at a startup - You can setup jenkins in a couple hours if you know what you're doing. Thanks to docker, it's 1-command to get it running. I'd then have nightly backups of it (easy via AWS or a jenkins job). Integrate it with LDAP so that you can lock down certain functionality if necessary (e.g. deploys, private creds). Set most jobs to have reasonable timeouts, retries, and alert the devops team on failures (devops will own making sure known-failure-cases emit the proper message and failure-code). Connect jenkins to splunk, and you'll have reliability and time statistics on every CI job.

Github PRs plug into jenkins, which can auto-lint, auto-build docker containers, and such.

Splunk (or sumo): Get logs / metrics / traces into here and/or datadog.

Widespread PHP at a FAANG has to be Facebook, right?

I've yet to see these mythical industry tools that do these tasks well. :(

You can say whatever you want about Facebook, but their tool/system for committing code changes is sheer perfection. Starting with managing source code changes on your own dev server, getting it code reviewed, merging it into the main branch, and getting it deployed to production. All of it is easy to figure out, and it works without complications. It can handle pull requests of 20000 files without any trouble.

Reply on news.ycombinator.com