Hacker News Reader

Comment by colinrand | Hacker News Reader

The most challenging part of this series is not in the material itself. These insights are learnings through hard experience and scale from Google are invaluable.

No, the hard thing for everyone is to recognize is that most companies are not Google and don't have Google's problems, resources, or time to follow these practices.

Definitely read the material, I will thoroughly, but don't apply this blindly. Solve YOUR problems, not theirs.

View on HN

We were very much aware that not all companies can afford to staff a dedicated security team. We tried to do our best to make sure that the book is applicable to a wider audience: from startups, to big corporations.

(disclaimer: I work at Google)

It's not as applicable to startups as you would think. The real calculation startups are making all of the time that this book doesn't mention is "is it worth making this particular piece scale/secure/robust before we run out of money?"

While it's technically true that the advice would apply to startups in the sense that it would improve their reliability, the elephant in the room is that it doesn't matter. The engineering skill at a startup is understanding what's actually critical, and this book doesn't speak to that.

How do you measure "is it worth" without a good idea of the risks and costs involved in the decision making? Just because it doesn't directly answer your question doesn't mean it's not applicable.

In fact, I'd argue the risk factor is significantly different across startups, so exploring the tradeoffs is the only way to approach the problem generically.

(Disclaimer: I work at Google and was involved with some aspects of the new book.)

The "is it doing X before we run out of money?" question is way overblown in startup land, usually by product people to skew developer time towards more features instead of much needed foundational work.

In reality, this question is almost always instantly answerable. You're either still building out your MVP and desperately need customers to validate your idea, in which case the answer is "No", or you're an established startup with runway and a growing customer base, in which case the answer is "Yes".

This doesn’t line up with my experience in startups. Security is never taken anywhere as seriously as all of the best practices (including this one) suggest. Same for cicd, etc.

Not to be dismissal - but that sound anecdotal.

I think it's best startups are provided with the most tools/options based on their priorities -- including the underlying lessons this book attempts to deliver - is the right path. Then it's up to their values and priorities.

Ignoring my startup experience (as they are all security-related and therefore took it serious), I believe startups that are handling any amount of customer data should be looking at security very seriously.

Now whether or not they do take it seriously is another problem, that doesn't mean the opportunities and advice shouldn't exist.

Not to be dismissal - but your experience is anecdotal and from the security industry and has no bearing on the reality of running a startup whose business is not security.

>I believe startups that are handling any amount of customer data should be looking at security very seriously.

What you believe has no bearing at all on the cost/benefits of running a business. In the current regulatory environment, leaking customer data in the US costs less money than losing one big customer for a b2b startup. Guess what that means when it’s time to decide to work on a feature for a specific customer or to do a full source code audit of all dependencies for vulnerabilities?

Best practice is the "best" practice, not the "most common" practice. The thing that sets "best practice" apart from "common practice" is that most people haven't actually done best practice; if they had, they'd just do it again, because it's much quicker and more likely to succeed if you've done it before. And money has nothing to do with implementing things the right way.

I disagree. There is a valuable question of "how reliable does this system need to be?" and for startups, the answer is often not 5 9s of uptime.

99% uptime is 14 minutes of downtime per day. There are an awful lot of processes and even whole businesses that can eat 14 minutes of downtime a day. Especially if it's not a full outage.

If your developers, at their core, have been building secure and reliable systems for years, then they can make the new system reliable and secure far faster than a team that always said “there’s no time.” It’s like every skill - you can go amazingly faster, or better, after it gets deeply ingrained after a few years

There is certainly a huge YAGNI danger here. 10 people shops should read this, but keep their socks on.

I love the part in that other SRE book where they say to “keep it simple” right after describing probably the most involved, meticulous and vast set of software engineering practices of the last 10 years.

“Simple” if you have 10B+ in the bank and 1000+ engineers to run the show.

This is why we need strong open source or liberally licensed components to build with. Small companies need to work together to keep up with the complexity of modern systems

Small companies just need to avoid getting into anything complex.

Both points of view seem valid, but patronising 90% of the programmers will probably decrease their overall happiness and retention. Also, some small companies do solve hard problems with no easy solutions both in terms of business logic and/or infra requirements.

Absolutely, but it's unlikely a situation like that will be improved by trying to emulate Google etc!

Yes! This is the reason IT as a whole is costly, time consuming, and complex: useful software products are not free. Software components are free; they'll give you the building plans and some raw materials, but it's up to you to raise the skyscraper yourself. It doesn't have to be that way. (It will be that way, though, because companies are pathologically terrified of giving a competitor an advantage by working with them, even if it's mutually beneficial)

Like Kubernetes?

People say this a lot as an excuse to keep the status quo, but often times the problems do overlap and the solutions are generally good everywhere.

You're absolutely right. I have heard that argument first had used as a means to shut down an initiative as well. These philosophies require massive judgement calls from engineering leadership with payback periods tracked in years, not quarterly OKRs.

The capacity of engineering organizations to successfully undertake multi quarter efforts is probably the best sign of competence. There seems to be a lot of short term thinking nowadays with projects based on quarters and those that don't bear fruit getting canned. Without long term investments, the org has to continuously put out fires which hobbles it and ultimately affects its ability to compete with other companies.

They don't mind long term investment in a product, but they do mind long term investment in improving their operational practices. Propose they spend 3 mil on a new team to build some new service? Sure, go ahead! Propose they spend 3 mil on a project to continuously improve practices that will maximize efficiency, speed up development for all products, increase quality overall, and reduce headcount? Unnecessary cost, let's just hire a consultant for 6 months.

I find it interesting that you think this might be a problem for some people. My experience would perhaps, if pushed to, lead me to conclude the opposite.

Reply on news.ycombinator.com