Hacker News Reader

Comment by wpietri | Hacker News Reader

On the one hand, I love this. There's an old-school, down-to-the-metal, efficiency-is-everything angle that resonates deeply with me.

On the other hand, I worry that just means I'm old. There are a lot of perfectly competent developers out there that have very little idea about the concerns that motivate thinking like this C10M manifesto.

I sometimes wonder if my urge toward efficiency something like my grandmother's Depression-era tendency to save string? Is this kind of efficiency effectively obsolete for general-purpose programming? I hope not, but I'm definitely not confident.

View on HN

It really just depends on how much of a computational surplus you have. Some tasks have become so incredibly easy relative to today's computational horsepower, that we can afford waste 90% of it if it means we can be 10% more productive.

But this isn't true everywhere. Imagine if Google's servers cost 100x more to run, if they didn't spend time to make their code efficient. Imagine if your video games ran at a frame rate 100x slower than they do now. Imagine if your phone sucked 100x more battery power, and lagged 100x longer after each tap.

Sounds like I'm making extreme hypothetical examples? Not really, when languages like Python and Ruby can be over 100x slower than other high level languages (e.g. Java, not to mention C/C++).

And of course, the counterpoint is usually "But most of the time is spent in the database/[some highly optimized library], not the high-level logic." To that I reply: You're absolutely right. Guess what those subsystems were written in? C/C++/Java, most likely.

If you're fortunate enough to have all your performance-critical components already highly optimized and written for you by experts at bare-to-the-metal efficiency, you probably don't have to think about this. But if you're working on a technology truly new and unique, chances are you're going to have to "get your hands dirty" at some point if you want to avoid paying 100x more than is necessary in server fees.

> when languages like Python and Ruby can be over 100x slower than other high level languages

You said a wrong thing. Java is a far cry from a high-level language.

Javascript is arguably a high-level language however, and meaningless benchmarks (aka "damn lies") attest V8 is in spitting distance of Java. http://benchmarksgame.alioth.debian.org/u32/javascript.php

Scala is much higher level than Java, compiles to the same run-time, and is at rough parity. http://benchmarksgame.alioth.debian.org/u32/scala.php

"languages like Python and Ruby can be over 100x slower than other high level languages"

Notice the "can be"? It's important.

And I don't think arguing if Java is a high level language or not is very useful in this context. Notice that C++ was also included as a "high level" language. Clearly if C++ is high level then Java is too.

Anyway, the point of that comment wasn't which languages are high level - it's that efficiency is important, because some languages have areas where performance suffers.

Clearly that is true (at least to some extent) as the use of Numpy in Python shows.

I'd argue that writing Numpy (and other high-performance libraries for use in high level languages) isn't "general purpose programming" (which is what the OP) was talking about, but that is also a different discussion.

What makes you say Java is not a high level language? It most certainly is that according to the official definitions of the term.

Javascript is arguably a high-level language however, and meaningless benchmarks (aka "damn lies") attest V8 is in spitting distance of Java.

There are far better examples. E.g. Haskell is very high-level and, as axman6 mentions elsewhere in this thread, has been shown to be able to handle 20m new connections per second.

We can revive the old 'language generations' concept I was taught in school:

- First generation: binary opcodes entered by hand.

- Second generation: Assembler languages.

- Third generation: Compiled languages like C++.

- Fourth generation: Interpreted high level like Python and Ruby. Also SQL.

- Fifth generation: There's no such thing. Or may be Apple's Siri is.

Java seems to be in a "3.5" generation, if that thing can be conceived. Javascript started as fourth and keeps getting a little bit closer to third.

Or, we can just call them all "programming languages" and evaluate them on their individual strengths and weaknesses.

I'm reminded of a recent article that crossed the front page of HN, about a group writing some sort of fluid layout engine for their mobile app. They were able to optimize it from tens of layouts to thousands of layout per second, which enabled their strategy of trying a bunch of different layouts and picking the best one. This wouldn't have been possible if they stuck with the naive solution. Making it fast enabled a new feature that simply couldn't exist with the slow solution.

I believe it's what used to be called 'craftsmanship'. Taking pride in creating things that are efficient and not wasteful for no other reason than the desire to make the best product possible.

A beautifully carved chair may display excellent craftsmanship, but if you spend three months carving the world's most beautiful chair when the client asked for a dozen basic ladderbacks for a dinner party next Friday, you're not a very wise craftsman.

Similarly, if your software runs in 1% of CPU on a typical customer's machine, spending 10x the time and resources to make it run in 0.1% is not laudable. Context is everything.

>Similarly, if your software runs in 1% of CPU on a typical customer's machine, spending 10x the time and resources to make it run in 0.1% is not laudable.

No, but spending 1.2x the time and resources might be. That's not an unreasonable proposition; something written in Go, C# or Scala can run 10 to 100 times faster than the equivalent code in Ruby, but it certainly doesn't take 10 to 100 times longer to write the code.

It also conserves resources; you're reducing the amount of the world's non-renewable energy that your app consumes by up to 90%. You're also freeing up your users' machines to do more things simultaneously while running your program, potentially increasing the user's enjoyment of the system in the case of a desktop app.

True! I'm just saying that good craftsmanship lies in understanding the situation and finding an appropriate balance. Ultra-efficient code is virtuous, but it's not the only virtue.

It is wasteful to spend time doing something that benefits nobody, or where the benefits outweigh the cost. Your time is worth something.

The point I was trying to make is that if you spend your time to make something that works really efficiently, and you take pride in it, then you haven't wasted your time. It doesn't benefit nobody, it benefits yourself, because you can take pride in your work. At least, I believe this idea is part of what's implied in the traditional notion of 'craftsmanship'.

An example might be working nine hours per day rather than eight to produce something that only uses 10% of the resources (and that extra hour is unpaid). For some people, that extra hour would be a waste, but for others, it would bring them greater satisfaction knowing they'd delivered a more efficient product, even if it brought them no extra revenue.

Except the difference at the margin is never just an extra hour of a day. It's almost always weeks instead of days or years instead of month. And that is fine when you are doing things for yourself and have no other requirements on your time, but when you're being paid spending two month on something instead of week just pisses everybody around you off.

I mean don't get me wrong, I'm working on a project right now that I'd love to do 'right'. Spend weeks doing a literature study to catch up on and really understand the current state of the art, then implement three or four different approaches to really get a feel how they work under real world conditions. After that I would try to pick the best aspects of those algorithms and try to write some really smart code that analyses the input and picks the best algorithm based on the data. Then I would tune that code until it is as fast, stable and memory efficient as possible and finally put a really slick UI on top of it. Trust me, nothing would give me greater satisfaction.

Unfortunately I got the project late last week, deadline is in less than two weeks and I have two other projects I'm working on in parallel. So I'm going to end up grabbing some off the shelf solution, tweaking it until it works well enough, wrapping it in up in some hacky shell script and make up for inefficiency by throwing more hardware at the problem. It's not optimal, but deadlines are deadlines, and I'll have a contend myself with the satisfaction of getting a job done well enough on time, rather than a job done perfectly.

> Except the difference at the margin is never just an extra hour of a day. It's almost ways weeks instead of days or years instead of month.

Maybe we're thinking of different things here. What I had in mind was more along the lines of for instance choosing Go or Scala instead of Rails or *.js and getting a 10x speedup at the cost of simply adding a few type declarations. There are situations like yours where deadlines leave little choice, but there are also situations where a bit more leeway is available.

I think both situations are highly context-sensitive.

If you're founding a startup in an emerging consumer market, you'd be making a mistake to use something like Go or Scala rather than Rails or Node.js, because your whole success is dependent upon finding out what particular combination of design, features, and experience emotionally resonates with fickle consumers' minds. That takes a lot of trial and error; anything that slows down that experimentation process is going to cost you the market. Once you've found the market you can get VC and hire experienced technologists to curse out your technology choices and rewrite it in Go or Scala or Java or C++ or whatever.

If you're CloudFlare and billing yourself as "the web performance and security company", however, building out your architecture in a performant and reliable language makes a lot of sense. You know your value proposition: you want to use whichever technology stack lets you execute against that value proposition most effectively.

Twitter got a lot of flack for being built on a "dumb" Rails architecture, but I'm also certain they would not have succeeded had they done anything else. Remember the context of their founding: Twitter grew out of an idea lab that grew out of Odeo, and at the time of their founding they were an idea that was so marginal that nobody would've bothered with it had it taken more than a weekend or so to prototype. When it turned out that people liked it, then they could afford to hire people to rewrite the software into something scalable - but those people would not have jobs had the initial concept not been proven out first.

You're right I suppose. I think as someone who's not a webdev I just have trouble imagining the productivity benefits of something like Rails or Node.js. I assumed that nowadays there'd be frameworks in statically typed languages with type inferences that were almost as productive as Node and Rails, and hence could provide both fast development speed and efficient executables.

You perhaps imagine that when Twitter was created the developers sat down and said "should we write this in Rails or in Java?" I very much doubt a conversation anything like that took place. I'm almost 100% certain the developers used the tool they knew best and never considered any other alternatives. Quite possibly they had little or no experience with alternatives.

I work with both a RoR code base and a Scala code base. There is no difference in productivity that I can tell between the two. I think it is a out-dated assumption to believe that dynamic languages like Ruby or Python are more productive than modern statically typed languages. The only cost is one of learning; many more people know Ruby than Scala. This is an artifact of history (i.e. Ruby is 10+ years older than Scala; crappy CS educations that only teach Java).

I think the "just good enough for business" attitude contributed to the demise of the American car and the rise of the Japanese ones. The American tradition was to use engineering tolerances that would maximize throughput under the constraint that it produced a pretty functional car. The Japanese tradition on the other hand was to use tighter tolerances, well because there was room for tightening the tolerance. At the surface level, or the MBA level it seems that the Japanese way is just dumb. It does not make monetary sense when measured in sales per year. But it turns out that the benefits express themselves at a different timescale: better brand and a culture of devotion to improvement, rather than to being "just good enough to work so that I can move to the next thing".

Part of this applies to the craft of software too. One can be sloppy and churn out functional websites by the dozens. At the superficial level, the goal to extract the most from a 500MHX Pentium-III might seem brain dead, with little or no pay off. But it pays back by instilling an attitude of deeply learning your craft, that learning does pay back although that specific well tuned web server might not. You dont have to build all your servers that way, but if it doesnt hurt you a little when you know exactly what you could have done to extract some more juice out of it, you will not reach that level of understanding. If you are impatient or sloppy, you will build impatience into your product, and it will show.

Besides it is always easy to over value ones time, it sometimes correlates with conceit.

@sitkack what makes you think I was talking about the 50s. Going by the downvote I seem to have touched a nerve. You are right Japanese products around that time were synonymous with bad quality, not just in US. Much water has flowed under the bridge since then.

Your recollection of Japanese manufacturing is clouded by mythology. Japanese quality sucked in the 1950s, they had a horrible world wide reputation for shoddy goods.

Maybe German manufacturing would be a better example?

German quality sucked some half century earlier, so much that England had to require them write "Made in" on their goods to warn consumers.

Quality comes and goes.

Exactly. When people complain about the quality of Chinese goods as a proxy for China and Chinese people as a whole, I ask them where their MacBook Pro is made and they quickly shut up.

All of these lazy, low tolerance sweeping generalizations need to go!

...which got better because Japanese companies had a culture of continuous improvement. That's very relevant to the discussion: the point the parent posters were making is that rather than settle for "good enough" you should continually move the bar for what "good enough" is, because then you'll eventually end up overtaking fat & lazy competitors even if they have a large head start.

I think you are actually exactly opposite on this.

There are actually two books that were written about this very topic, one called "The decline and fall of the American Programmer", which was essentially making your argument, that American quality going down meant we would get eclipsed, and "The Rise and Resurrection of the American Programmer", written by the same author, saying "oops, I was wrong", because "good enough" is the winning strategy, not quality at all costs.

Note that at the time of those writings, the 'sloppy' guys were Microsoft, which is still way way more process than your typical RoR shop today has. In the days of shipping new software to websites three times a day, pragmatism rules all.

As the saying goes: "Premature optimization is the root of all evil."

I think you are conflating the demise of the American car, "just good enough for business" and the fetishization of over tolerancing. All of those things are much more complex than "a tradition" of quality. I would assert that Japanese quality is a rather new thing and that it ebbs and flows with the time and the needs.

By your measure, the Zero was a low quality plane, but served the MBAs in the Japanese military well for meeting its objectives. The AK47, one of the most successful machines in the world has extremely sloppy tolerances and it is exactly those lenient tolerances that enabled its success.

We always have to be cognizant of economics, over building, polishing or engineering a system is a waste. Calling it a craft doesn't make it any more acceptable. Proing-up and being good at what you do doesn't mean you need to put burnished wood knobs on your software.

As far as AK-47 goes I think you are conflating two different engineering concepts, perhaps because you are not an engineer. One concept is that of "clearance" as a feature for the designed functionality of a product vs loose tolerance as a result of sloppiness.

I think, you think tolerance means fits that rattle, thats not what it means in engineering. Loose tolerance may just as likely lead to interference fits.

Forget AK-47, there are more egregious examples, consider the Blackbird, SR-71 one of the fastest military aircrafts to have graced the skies, a marvel of engineering: it had just wide gaps between the parts of its metallic skin, at the joints. Now going by your logic one might think SR71 became successful because of loose tolerance and "good enough". It was quite the opposite, the clearances were deliberate and necessary for its success. They were there to account for thermal expansion. For the AK it was necessary for rough use and low to zero maintenance.

In fact not having clearance for SR71 or the AK would have constituted intellectual sloppiness: going with cookie cutter decisions and not optimizing the product for the use case. What looks sloppy to an amateur might actually be the result of perfection by an expert.

May I recommend Zen and the Art of Motorcycle Maintenance to you.

Further, I think you are being obtuse and defensive and railing against something I have not said. I am not quite sure why. I have never claimed all products need to be finished to the point of "burnished wood knobs". My commentary was on personal growth as an engineer. If it doesn't bother you somewhere to turn in a product that you know you could improve with little effort, you are not going to be a quality engineer, and will not be able to produce a quality product when one such is desired. I am categorically not saying that each item that you deliver has to be the epitome of some arbitrary quality standard.

The cultural tradition that I was talking about was not about adding cost to the product by unnecessary finish. Often people do not finish the product even when it would not have taken much effort. This is rationalized with the logic that the finish would have little immediate value, because it is already good enough for the job, but far from "good". The other reason is sometimes the craftsman just does not have the skill and "good enough" is a good argument to take cover under.

You keep using that word. I do not think it means what you think it means.

Is this kind of efficiency effectively obsolete for general-purpose programming?

I think it is obsolete in that area, and has been for years (decades? Unoptimized SQL is almost always slower than key-value stores). General purpose programming is really about meeting user & business requirements. Speed and ffficiency is a second level concern, which only becomes an issue when you don't have enough of it.

BUT efficiency is even more important now for systems-level program ("system" in the sense of building libraries for use by other systems, as opposed to user-facing).

A good example is SnappyLabs[1] who developed a faster JPEG encoder and ended up being a subject of a Google/Apple bidding war.

[1] http://techcrunch.com/2014/01/04/snappylabs/

I think there has been a renewed interest around 10Gb ethernet (and SSD). Basically computers were grossly limited by IO with gigabit ethernet and hard drives, you could buy a pretty low end box and saturate them, instead people bought high end boxes and ran slow code. But your average slow code wont perform well on 10Gb.

Indeed. I've got an architecture right now that I could actually do everything I need to do in a single rack with 40GigE adaptors, except...I can't (easily) get that kind of bandwidth to so few boxes using third parties, and more importantly, I can't get the bandwidth cheaply that way.

So instead, I'm scaling out with mostly idle Intel E5-1620v2 4 core (8HT) boxes and single 1GbE connections because that's about 70% cheaper for access to the same bandwidth. In fact, bandwidth is actually the only cost we even monitor anymore, because that's what we're limited by (the machines come with far more bandwidth allocated than adding that same bandwidth to an existing machine, so the machine is essentially free).

So now we're swimming in CPUs, RAM, and SSD storage, and we're renting—no joke—the cheapest possible hardware we can at our provider (OVH). And no, we're not serving images, audio, or video—our CDN handles that (although see below for why I dropped S3). I actually do public-key authenticated encryption on every single packet and I've still got tons of CPU to spare.

Honestly, I feel a little bad about the whole thing. Why should I use 600 boxes when I really could do it in about 12? It just feels incredibly wasteful.

To partially make up for it, I put the 2x3TB of rusty metal storage we get with each of those tiny 600 boxes to good use, by moving us off S3 (which'll save another $100K/year, and a lot more as we continue to grow), so it's not been a total waste. Maybe I should look into Bitcoin mining next? :)

> But your average slow code wont perform well on 10Gb.

Yup, that really does require looking at things differently, measuring things differently. You actually need to have some kind of mechanical sympathy to remain efficient. Or build off of a project that does that for you, like Snabb Switch[0].

[0] https://github.com/SnabbCo/snabbswitch

Not necessary if you can displace your own purchases, but my first thought was that maybe you could do something like "Amazon Glacier", offering distributed but rarely used storage for a fee. I presume that Amazon offers this because they have a similar surplus of live empty disk space.

I know you are joking about the Bitcoin mining, but my second thought was that maybe there is a parallel 'proof of storage' idea. There could be a public market for backup storage, where instead of being paid for 'proof of work' one is randomly tested for 'proof of backup'. You say you'll store something, and you are paid based on your ability to answer random 'challenge' requests in a timely manner.

And then I noticed another front page article on HN from someone writing software that could do something very similar: http://hypered.io/blog/2014-02-17-building-reesd/ Perhaps you could be his backend.

Having dealt with applications spanning thousands of machines, I'd say there will always be value to optimizing programs on a single machine. The prerequisite to distributed computing is being able to program a single computer.

If you can save an order of magnitude in machines, you will always avoid some scaling problem. 10 machines, 100 machines, and 1000 machines all have different problems.

Not at all. I think that it's our perspective shifted, because our every day computers became so fast, with more CPU and more RAM, that the average programmer doesn't feel the pain of inefficient code anymore. I've seen enough software that is "very fast and scalable", because it wasn't tested at really large deployments.

Of course, that's not always what your software needs or wants, but in the case of internet backend software and hardware, it's a huge difference if your servers have 1, 10 or 100 times the throughput.

Also, many developers are client developers where this is a not the problem it used to be. In 1990, when a game didn't fit into a few kB of memory or didn't ran fast enough, the developer assembler-optimized the hell out of it, today most of this doesn't matter except for the software everyone uses but noone writes (your ISPs webmail, dropbox, twitter and whatever internet site has enough users).

As someone who learned to program on a machine with five kilobytes of RAM, and later had to put a fair bit of effort into unlearning some of the resulting habits:

Yes, that kind of efficiency (in terms of machine resources) and inefficiency (in terms of human resources) is effectively obsolete for general-purpose programming. What of that? If your heart's desire is to work in a context where machine resources are scarce, just work in an area like high-performance technical computing where that's still true. We do not, and never will, have any shortage of jobs where you always need more computing power.

VIC-20 syndrome - I likely have it.

There's plenty of room for both. Many useful, worthwhile applications won't ever really push a machine to its limits, rather they primarily involve bringing structure and order to mounds of complex business logic. And that's OK. Not everything needs to be optimized.

On the other side of the same token, many useful, worthwhile applications will absolutely depend on this level of optimization. There are some problems which simply can't be solved in a practical manner without it. And that's OK too.

Most of the time your time is better spent adding feature to an application then trying to tune performance in the ways he talks about on the site. Most of the time it is easier to scale across multiple machines and pay for the hardware then it is to put in the time to make a program this efficient.

But there are applications like Snort (which he seems to reference a lot) that can't be split onto multiple boxes easily, so tuning for max performance is very important and is a key feature of the application.

Reply on news.ycombinator.com