Hacker News Reader

Comment by poulsbohemian | Hacker News Reader

We're bad for several reasons:

-- Outside FAANG type companies, most businesses are not willing to actually put in the investment it takes to do engineer software. They want a "solution" as cheap and fast as they can get it. I'm not even passing judgment here, as in some cases that approach is entirely justified.

-- Software is a world of eternal September, where the industry memory is only about five years long. Lots of us move on to other fields or roles and the memory is forgotten, and here comes another generation to reinvent the wheel all over again.

-- Education: Computer science is not software engineering. Code schools are teaching programming more than software engineering. Thus, software engineering is something learned through doing rather than something studied as a discipline.

View on HN

The competency of FAANG is completely overblown. Mostly they enjoy massive profits by being monopolies. Because they're monopolies, they can afford to waste a ton of money on software. Facebook is not even known for high software quality, theyre actually known for speed and getting the work done as fast as possible. Same for Amazon.

Sure, they don't have to be efficient so they can afford to spend more on tool development. Not everything works but the better approaches get more adoption.

Also, I think you're underestimating the effects of having a lot of code. This pushes limits that smaller companies don't have to worry about.

Elite startups and hedge funds have the best software quality. There are funds in NYC where every programmer makes more than 500k a year, some in the millions. They have people on those teams who have been programming since they were in middle school. FAANG just hire in bulk, they just want you to memorize solutions to algorithms problems, and if you can do that youre in. Some huge percentage of programmers at Google have less than three years experience. They can absorb all the bad engineers they hire, a few end up being good, who they promote to the teams actually writing the software critical to the business. The rest just write CRUD Apis and shuffle data around, using some framework written by the top 1% in the company.

Surely there are some teams in the heavily-regulated aerospace and medical technology spaces as well. NASA would apply.

If you haven't been writing code since middle school, how would you recommend getting into those funds?

It’s a weird world. I worked at D. E. Shaw and their initial phone screen includes SAT score. I was 26 at the time and had to ask twice to be sure I’d heard the recruiter correctly. For whatever reason, that was part of the process. They were also the origin of the “every new hire must have an articulated dimension in which they are better than the median for their role and level” that Bezos later took to Amazon.

That was the most consistent top caliber colleagues I’ve ever worked with. (I’ve worked with other excellent colleagues, but Desco was uniformly very good. Our office manager had a PhD. Our recruiter had a masters degree. It was crazy.)

Probably the easiest path is to be referred in from a current employee. Next would be outstanding achievement in some academic or technical field (if applicable). You don’t have to be an autodidact or a savant. You need to be competent, intelligent, and willing to work hard and then to find a way to get a warm intro to recruiting.

One of those funds famously blew themselves up in 45 minutes, with causes including reusing an old feature flag rather than defining a new one and a complete lack of code review or incident response procedures. In all my (admittedly anecdotal) knowledge, it may be true that the top funds have the top software engineer quality, but their skills are put to use writing software that's absolutely terrible.

> Elite startups and hedge funds have the best software quality.

Most startups that I hear about have rather shitty software quality because they're focused on velocity. VCs don't care about sustainability since they don't even know whether the business is even worth sustaining.

Sure, but a truly world class programmer writes better code in a rush than a mediocre one does in a slow enterprise environment. Programming skill doenst work like that, some people are just better.

Not entirely true. Facebook, Google, Microsoft, Netflix, etc... They all wrote (or greatly contributed to) a bunch of high quality open source projects that you probably use on a daily basis.

Yeah they did. But its a small fraction of engineers that wrote those libraries. Not only that, most of the engineers doing foundational work at those companies are not getting hired through standard interviews. When Google wants to hire people for a team to write a new distributed database, they arent asking a bunch of leetcode questions to engineers. They're hiring out of PhD programs and from accomplished teams across industry. People do get pulled onto those teams from other teams in the company, but it's only after those people have proved themselves. As a whole, only 1% of the engineering at those companies is what you are describing.

Look at the white papers for the foundational systems at Google and FB, most of them are authored by the same few teams. Jeff Dean had a hand in designing a ton of large scale Google systems, just because he came up with some brilliant solution, does not mean Joe programmer at google who can implement A* in 30 mins is in the same league.

I'm curious, what evidence do we have that the FAANG type companies are significantly better at software engineering than anyone else?

When we look at the entirety of companies around the world, it's reasonable to think that there are non-FAANG companies that could be just as skilled. That stated, my own casual observations:

1) The FAANGs that I've seen have significantly more robust practices than the non-FAANGs. Everything from office conditions to build / test / release systems. The reason for this is:

2) Incentives. When your business is completely focused around software technology, you are incentivized to build out systems and processes that other places would not, because it is core to your business.

Anecdata from many years in and out of various kinds of companies, as both an employee and/or road-warrior consultant.

I still see bugs and rollbacks in FAANG releases. I conclude that the incentive isnt enough. The practices are inadequate.

A process with occasional bugs and rollbacks is rational if it produces more net utility than a process which only ships "perfection".

What's your error budget? Not spending your error budget is velocity left on the table.

That's... like the whole point of the article. That mentality causes less reliable software.

This is my point exactly, they arent. They just have monopolies.

Well at least they are trying to be. Some of them employ PhD's in theorem proving to apply computer assisted proof to improve their software (and hardware).

Who's doing this? Amazon is using model checking on simplified models of real-world software systems, that's not "theorem proving". Microsoft is doing some related stuff in low-level code (Sing#, etc.) but they're not in FAANG. Galois is not even close, they're a relatively small shop.

The other A is doing it.

To accompany the neighboring reply from the grandparent poster, the practice of site/service reliability as an engineering discipline is not just popular amongst the FAANGs, all of whom have SREs, but embedded into the company culture and structure. Google, for example, has a head honcho of SRE (a VP of SRE?) who gives all SREs extra authority to act somewhat independently.

A large part of the perceived competency of the FAANGs is their ability to casually and repeatedly achieve 5+ nines of availability. When HN is unavailable, that's no big deal; when google.com is unavailable, users act as if an apocalypse has begun. That difference in reliability changes a service from something that people consider as an option into something that people rely upon.

To your first point, not even that alone.

People aren't paying (only) for stable and correct software. They actively seek out fancy UIs for example. I noticed that in a former company. We had the best algorithm and the most correct numbers (for what we were selling) but the customers didn't like them because they were lower (and more realistic) than some competitors' numbers. Also they had better and shinier UI, they won the market.

There is plenty of very buggy and very bad software from FAANG companies. I'm not sure that first point holds too well.

are you suggesting that facebook or google's or MS products are the best engineered? I was just using skype ...

Skype is a bad example because it was something that Microsoft took over.

We're gonna have to exclude some huge percentage of Google's product portfolio if that's the standard, then. Though much of that is pretty bad so maybe you're on to something.

it was good when microsoft took it. they made it worse

https://www.howtogeek.com/369631/microsoft-is-killing-skype-...

btw the list is much longer: twitter, facebook, adsense ... visible bugs in all

Reply on news.ycombinator.com