Hacker News Reader

Comment by Animats | Hacker News Reader

The author's acronym is silly, but it's a real problem. Soylent liked to blither about their "infrastructure", for a product that sells a few times per minute. They could be using CGI scripts on a low-end hosting service and it would work fine.

Wikipedia is some MySQL databases with read-only slaves front-ended by Ngnix caches and load balancers. That seems to get the job done. Wikipedia is the fifth busiest web site in the world.

Netflix's web site (not the playout system) was originally a bunch of Python programs.

The article mentions a PostGres query that required a full table scan. If you're doing many queries that require a full table scan, you're doing something wrong. That's what indices are for.

View on HN

In fairness to Soylent, their marketing revolves entirely around the perception that they're a tech company and not one of many meal replacement shakes. The problem they were solving might have been "Techcrunch hasn't written an article about us recently", which an over-engineered stack does actually fix.

Yeah, you can't buy that kind of publicity.

Wait...

I remember reading a scaling-out article from some startup. Some of the things felt a little over-engineered, some were impressive, some seemed wrong. But then they get to the point where they brag about their scale, and the metric they used was that they can handle thousands of requests.... per day.

Be honest, were they hosted on their founder's smartphone?

Great idea! I should try this for my next project ;-)

https://engineering.hellofresh.com/scaling-hellofresh-api-ga...

...with only 10+ high-CPU large machines.

With that level of traffic, they could run their entire frontend from a smartphone...

Why did you have to type that. Now in my free time there's a 50/50 chance I am going to waste minutes or even hours looking into the viability of running a postgres/sinatra/nginx app on a phone.

Something about that appeals to me in a huge way. Showing people my "server room", etc etc.

Attach the phone to a small weather balloon, and you can have a real "cloud hosted" setup.

Plot twist: Run it from a flip phone.

There's a bunch of httpd apps on the Play Store, many complete with database engines and all that.

Phones are expensive. Raspberry Pi.

ergh, you weren't kidding. I hope that was a typo on their part; even low thousands rpm is mostly not a big deal on modern hardware.

   Our gateway is on the frontline of our infrastructure. It receives thousands 
   of request per day, and for that reason we chose Go when building it, 
   because of its performance, simplicity, and elegant solution to concurrency.

For all we know the blog is a work of fiction and they're running off a laptop...

Holy shit! I almost didn't believe you.

Then again, if they didn't have that beefy infrastructure, they could get DDOS'd from up to multiple request per minute! I mean you can't be too careful when you are in that league!!1!

Also interesting (in the same way as rubbernecking a car crash) is going to their homepage and viewing source.

I didn't want to share the original article, but you found it :) Then again, Google search doesn't turn up many other results bragging about thousands of requests per day.

'Dashboards around the office show how the system is performing at any given time.'

Probably was someone's full time job for a couple months.

When was this written? Why is there no year in the date?

Granted, HDMI can be infuriating, but months?

More seriously, you can set this up with KPIs in a day with Zabbix. A week, if you've never dealt with Zabbix, which is admittedly somewhat complex compared to some of the others.

Depends on what you're trying to accomplish.

But setting up a data warehouse that builds out the KPIs you're looking for along with a dashboard to display them (the easy part, for sure) can certainly take over a month.

I am surprised they are still up, given the amount of attention from the HN community ;-)

I think their CDNs are saving their 10+ High CPU servers from the load.

This is hilarious and sad at the same time. However, most of these write ups are aimed at attracting talent. Even more, some tech stacks are deliberately built to attract talent when the core domain is just too simple or boring. "We serve user subscriptions and recipe data from an SQL database using Rails" just doesn't sound as snappy as the infra-porn on the blog.

Your attitude frankly sucks.

How can it be a bad thing for developers to write about what they are working on ? Not all developers are genius experts who suddenly woke up one day and instantly knew how to perfectly architect a system. People learn, people grow, people make mistakes and most importantly people need feedback.

Blogs like this are fantastic to (a) learn what other people are doing and (b) for developers to have an opportunity to show off and be proud of what they are working on. Especially since the rest of the company in most cases isn't going to care less.

It's great to write about your tech. But unless blogging is your business model, it's not great to pick your tech so you can write about it.

Isn't that kind of thing a real red flag for the kind of talent you'd want to attract? If someone told me they'd built a GPU compute cluster for their phpbb based social club forum, I'd think they were an idiot and not want to work with them.

Absolutely. I'm running into this problem constantly in the front-end dev world. Most developers I talk to are way too eager to list off a million cool techs that they're familiar with, most of which are completely inappropriate for the job at hand.

If I'm interviewing a candidate, I'm not interested in which tools they know how to use. I'm interested in whether they understand the problem those tools solve and when to apply them. Over-engineering is problem #1 with the community at large in front-end dev at the moment and you'll go a long way as an engineering team if you make it your priority to defeat that mentality.

I am sorry if I'm completely off base but I'm still thinking about the danluu page on options v cash which quotes this

https://news.ycombinator.com/item?id=11200296

Maybe startups don't want the absolute best of the best but rather the best of the gullible?

Edit and or poor

So you're thinking that kind of technical mark-missing is the startup equivalent of the typos and other glaring errors in email scams? They're to weed out the people smart enough to be a problem?

If I'm ever in a position when I'm needing to attract talent, that would not the kind of talent I want to attract.

I'd want grumpy, paranoid sysadmins who really don't want to be woken up late at night because something has gone wrong.

Hey, that was a major milestone for a RoR app.

At the other end of the scale, an acquaintance of mine is a sysadmin for a government department, and can't get their web framework changed to his frustration (politics...). He gave an example of the websites he has to look after, one being a boring legal website ("so you can imagine how low the traffic is already...") and the software could barely serve one request per second!

> Wikipedia is some MySQL databases with read-only slaves front-ended by Nginx caches and load balancers.

You're not wrong, but they are slightly more complicated than that: https://meta.wikimedia.org/wiki/Wikimedia_servers

They have Kafka, ElasticSearch, some workflow engine thing, and they manage a distributed object store (Swift) deployment.

Apparently in Wikimedia's case they also have a stated company policy / philosophy around self-managed infrastructure that prevents them from using public cloud for a bunch of stuff (I've heard this from a former employee but can't find any further documentation about it).

>Netflix's web site (not the playout system) was originally a bunch of Python programs.

seems that the bunch has grown slightly since then:

https://cdn-images-1.medium.com/max/600/1*OhONoCsb1KvesLy-n5...

(from https://medium.com/netflix-techblog/announcing-ribbon-tying-... )

Or this view https://image.slidesharecdn.com/devopsisraelpdf-141030000609...

(from https://blogs.mulesoft.com/dev/microservices-dev/api-approac... )

This happens all the time on StackOverflow/ServerFault. "What sort of server cluster do I need? I have thousands of hits a month!" Usually the answer is "an average smart watch could do it".

Yes, forget the acronym.

It's simply, "think for yourself." Do you ever talk to someone on a topic (in technology or politics) and it feels like you're talking to list of commonly accepted prepackaged talking points?

> Netflix's web site (not the playout system) was originally a bunch of Python programs

That is fascinating, I would love to read up articles about this.

Queries that require full table scans? What are we talking about? WooCommerce?

Hey! Those all have indexes now. :)

Good to know!

Working on a migration from a site that modified the WooCommerce core to a new ecommerce platform. Couldn't believe WooCommerce ever did that when I started.

Congrats on such a widely used platform though.

Wikimedia actually uses Kubernetes, but not on the serving path (yet?):

https://kubernetes.io/case-studies/wikimedia/

Kubernetes is essentially platform as a service. The parent didn't address the backend infrastructure,only the services serving the web content.

That's just for the Tool Labs (tools.wmflabs.org).

Reply on news.ycombinator.com