Hacker News Reader

Agreed. In the other story about node.js I challenged people to describe the "Node way" of doing things, and was given a laundry list of things that applied to every technology, like "modules".

We are currently living through a programming language and tools renaissance the likes of which have never been seen before. Why are good developers deliberately blinding themselves to all the goodness throughout the whole programming ecosystem, choosing instead one tool (e.g. nodejs) as the One True Hammer? It's so unnecessary and so limiting.

Interesting choice of words renaissance, as what we are actually seeing is people blindly reinventing things that were known in the 1970s. At least the true renaissance thinkers knew they were rediscovering things that existed before the Dark Ages; today people genuinely think they're onto something novel.

what we are actually seeing is people blindly reinventing things that were known in the 1970s

Or people who have worked exclusively in web development and for a relatively short time suddenly discovering general programming knowledge, like why modularity and separation of concerns are important, or that being able to write prototype code and get it into production quickly doesn't necessarily mean that code will be maintainable or support the same pace of development over the long term.

Next week, if you do everything in a dynamic-everything language using a dynamic-everything framework with everything looked up at runtime from a database, your software will be slow. The idea that a simple blog site running on a modern server should have to worry about hitting the front page of an aggregator and then falling over because it got a few thousand visitors in the next hour is rather absurd, if you stop and think about how much processing power and raw bandwidth these modern systems actually have available.

While it is true that the basics of computation did exist in the 70's, it's very clear that programmer productivity today far eclipses what we were doing even 20 years ago. I started coding professionally in the 80's. Today much of what I used to do is now handled by users on their own with spreadsheets and query tools. A small team of 1-2 people today can routinely take on a task that would have taken teams literally 10x as much effort back in the day.

For example, anything involving interfacing computer systems together in any way took an order of magnitude more time back in the day. There were no real widely accepted, simple to use methods available. Networking and communications over modems tended to be so unreliable and difficult that systems tended to be made as monolithic things that just did everything for themselves.

It occurs to me that a book could be e written describing why and how programmer productivity is better today than it was a couple decades ago.

It is true that computer technology has been around a long time. And it's also true that a lot of the best stuff such as Lisp has been around for a long time. But there are all kinds of factors that make programmers complete projects in a matter of days today that they wouldn't be able to 10 or 15 years ago.

And yet, an incredible amount of software was written on VT100s. Software that is rock-solid reliable, does things on old hardware that would be impossible on even on modern hardware with scripting languages, and is still relevant and doing useful work after many decades. What looks like "productivity" today is just fluff, pretty interfaces that still offload their real work to old-school systems.

Get off my lawn!

I've used a bunch of different languages over my career from device+graphics libraries in assembly to windows apps to servers, web stacks and (recently) front-end. These days admittedly I do a lot of node development and while not a huge fan, its really so easy to get something up and running all the while using all the same tools and even the same modules on the front+back ends. When I run a build its the same gruntfile for the stack and UI, there's no fussing about getting disparate servers, paradigms and teams playing nicely and its small enough for me to get an entire application installed and running literally within a few minutes even on resource limited hardware like a rasberry pi - that creative freedom is brilliant! I can definitely rant about all the things that are bad about nodejs but as a prototyping tool or sideproject toy its really valuable.

> Node will not continue to be 'new technology' for long. This is a shoddy basis on which to select a tool.

Not to worry - there will be a new hot technology within a year or two, and a good percentage of the node.js people will scramble to rewrite in it to avoid looking old fashioned.

I know what you mean but I really think it'll be hard to topple node. I started with express and tried a few other frameworks and finally decided on sailsjs. I haven't had this much fun or been this excited about web development since I started 10+ years ago. I really don't know what it is about node thats so appealing as Im no js guru but I think its going to be around for a very long time.

Whenever a new technology/language/framework comes up I often hear the word "fun" all the time. My question is: does that fun translates into reliable infrastructure and decent optimized code?

What makes you think that JavaScript will still be the flavor-of-the-day preferred by this crowd tomorrow, though?

They were pretty gung-ho about Ruby and Ruby on Rails between 2005 and 2010 or so. Then their hype moved to JavaScript and Node.js quite rapidly. It'll very likely move on to something else soon enough.

The difference between Ruby/Rails and Node.js is that javascript is literally used everywhere on the stack. Such a HUGE volume of open source code is written in JS. Ruby/python/Java/C# have not enjoyed that advantage.

But that doesn't give any meaningful advantage. Want to do your database queries to the client? Your business logic? No. Want to put frontend engineers to write your backend?

Lots of bad ideas. I mean, it is an ok feature, but by no means a killer-feature really. Ocsigen does the same thing: you can write your code in OCaml and the client-side part will by translated to JS automatically. Neat? Yes. Killer-feature? Probably not.

And JavaScript is really not that great to start with, so daveloping in something else server-side is not the worst idea.

The powerful use of JavaScript for me is that I can render the exact same templates on the server and the client. That's a lifesaver when you want a fast, client-based app that is also search engine indexable.

And JavaScript is really not that great to start with

It's fine. IMO, all languages have warts you have to work around, once you know them it becomes a non-issue. I can't think of the last time one of those "JavaScript WTF" moments actually tripped me up in real code.

I guess what I mean there is less let's have front end devs write our backend but that the ecosystem is so huge and that so many people know JS that it's a huge advantage. Nearly every single web developer that's been working over the last 20 years knows some level of javascript. No other language enjoys that advantage.

The Language, (in this case JavaScript) is probably the smallest part of domain knowledge needed to be a good front-end or back-end engineer. Front-end engineer, know JavaScript? OK, optimize these queries, do we need to cache, make temp tables, map/redux against the DW, etc? Hey backend developer, know JavaScript? Why won't this render the same in Chrome 33.1 as it does IE 10...etc...

JavaScript is a convenience language across the stack for when the team is small, or even a single person.

yup this comment trail is pretty dead-on.

The problem with web dev (I've thought about this a lot) is really that the browser is kindof the hinge. So no matter what you do for unifying languages, even porting modules so they're same on node front to back-end... you still can't really ever get to a point where you're unit testing pretty reliably through the full-stack.

So might as well test parts in isolation & who cares if there is one language throughout the whole stack as long as your devs are good at the languages required.

One day I think the impedance between server & client will disappear, but only when the browser's role as a rendering engine diminishes or finds better integration/intimacy with the codebase.

(and incidentally, i stopped caring so much about this struggle once i learned how to develop pretty reliable SOA. Haven't totally given up trying to conceptualize the perfect stack tho)

I know enough Javascript to make something work. I know enough about Databases / Python / Perl to make it work, and make it work properly.

> Such a HUGE volume of open source code is written in JS.

That's because there is a separate module for everything! Want to run "mkdir -p"? There is a module "mkdirp" just for doing that.

If you implemented every command as a separate library, of course there is going to be a lot of code out there.

JavaScript is like Gravity on the client side, and with ES6/7 it's about to become a much nicer language too. You could never run Ruby natively, in a widespread manner, on the browser. With one language you can now build a complete stack and target all devices. That's quite big.

I couldn't disagree more with what I assume you are implying here - that node is just some trendy thing that people are only using to stay hip. I have a fresh perspective since I have only been programming (mostly web dev) for a few years and I've been learning on my own with the goal of full-stack competence. I did not care about what was hip, I cared about what would work using my limited resources. With that goal in mind I have spent my time practicing, testing and analysing most of the popular languages and related frameworks from asp.net, php, python, ruby and so on, and from my beginners perspective (which is a important market to serve) node is on another level. I'm fairly confident it is one of those big leaps forward like rails was and will be around a very long time.

Why? In one word I'd say its speed. Its freaking fast and I'm not just talking about io performance, thats just the icing on the cake. Learning curve, development time, configuration, all extremely quick and easy.

One thing I think people forget or don't put enough emphasis on is that node is a networking library, its not just server side JS the way that asp.net is server side C#, its JavaScript from the bottom to the top. Web server, load balancing, pretty much anything you need can be written in just JS. That makes it extremely easy to set up your own stack that you actually have the potential to comprehend or even write yourself and launch something on the cheap. For 5 bucks on digital ocean you can set up a node app that will probably scale enough to survive a slashdot effect or launch a mvp.

Node and its related frameworks still have a ways to go but I think it is a mistake to dismiss it. And finally I'd like to mention that its first release was 2009 and people have been talking about it a lot ever since. I think its already passed the trendy technology phase and will continue to see widespread adoption.

If you say "full stack"... then yeah, you care about what is hip.

All of this. NodeJS is a valid tool, yes, but not the sole reason for those performance improvements.

Take the PayPal case, which (iirc) is mostly based on a 10 year old Java stack. If they were to create a new project from scratch, using current-day Java technologies and frameworks based on current-day paradigms (technologies like Play, Akka, Vert.x), they would get equal if not even higher performance improvements.

I don't know how developer productivity and happiness would be using those tools, but the Java development tools are very mature (compared to JS), and happiness... I don't know really, I'm inclined to believe a lot of developers use NodeJS because it's new and hot and Java isn't, instead of looking at their customers' needs.

> and happiness... I don't know really, I'm inclined to believe a lot of developers use NodeJS because it's new and hot and Java isn't, instead of looking at their customers' needs.

I agree. To be happy, We don't program using a particular language. We play video games instead. We program using a language chosen by us or for us to satisfy customer's needs.

As much as I like dynamic languages and would like to find good reasons to support them, mainly python and javascript, I have to agree with this. I know there are issues with them but as a fairly level field bench mark the Tech empower tests show node.js is usually firmly middle of the pack as far as raw performance. http://www.techempower.com/benchmarks/

Moving from rails to almost anything could be faster.

Especially under load. Especially if you had time to fix all those old nasty bugs.

Rails is plenty fast - for application development. And a well-designed rails app, appropriately tuned, can be suitable for all but the highest volume loads.

This rails bashing is becoming a cliché, so here's another one - use the right tool for the job. I've worked on rails projects happily serving 100req/sec with the servers running at 20% load. I've got friends who work on J2EE sites which take 10 seconds to render a page after the app server's done talking to all its zillion little useless ESB friends. Is ruby faster than compiled java?

I currently work on a combined rails/node site - we use a rails API to feed a node-served front end. Works beautifully. Why is the API using rails? Because in my opinion, rails is far more mature, and one rails API dev can keep up with three node front end devs. Would the decision be the same if speed and efficiency on the server was of topmost priority? Probably not, but it wasn't. Hell, the DB is the performance bottleneck anyway.

I have nothing against node and will probably use it in an upcoming project where server effiency is the topmost priority but geeze, right tool for the job!

And another point, while I'm ranting - I am coming to see node as the obvious choice for consumer (free) and rails as the obvious choice for B2B/SAAS (paid). Why? Because hyper-effiency is much more important for the former to keep cost per user down. With B2B, if you've got so many paying customers you need another rails server, you throw a party!

Oh, and even the node devs I know deploy using capistrano. There's that right tool for the job thing again.

Thanks for the reply. This is almost exactly what I wanted to write, especially in terms of programming costs of Rails vs. nodejs.

I am working on a project with exactly the same setup now, Rails for the API and a NodeJS Proxy for the web – works beautifully and with the mature ecosystem around Rails, development of the API has been very easy and focused.

Much of the critique regarding the slowness of Rails relates to earlier versions of both Rails and Ruby, but nowadays it can match the most mature frameworks.

Rails is fast, if you've never dealt with anything in HPC.

Also I was more referencing that if you rebuild an app you expect it to be faster.

However:

100 Requests a second and at 20% CPU. Really!?

Considering basically all rails/node.js/php/etc apps do is add frilly bits to data held in a database, there is no real reason why it should be slow.

To put it into context, say you are serving a home page, its maybe 150k of HTML/JS of which 90% of it is the same regardless of who visits the page. (hence why proxies are so effective) Thats 15Megs a second.

Not exactly Uber fast, considering its effectively a dumbarse file server.

To put it in context, your standard linux fileserver will (over NFS) push out 1.1gigabyte a second at 30% CPU (assuming disks allow) From a ZFS file system (which is computing the checksums of each 4k block....)

so yes, rails is slow.

You know, when we say "X is fast/slow" we usually are basing the comparison on something even remotely related. If we don't do that, the comparison is basically useless.

Of course rails, and almost any other software, is "slow" compared to HPC. I struggle to name two less similar applications. You might as well say that aircraft carriers are slow compared to F16s - well yes, yes they are.

The rest of your comment indicates to me you have no experience with web programming. Of course the cacheable parts are simply read from a disk, or even memory. It's the uncacheable parts that are the problem. And no, it's not just "adding frilly bits to data held in a database".

You seem to have a bad case of "shit's easy syndrome" - the tendency of people who have no idea what they're talking about to assume that everything is easy, and anyone who disagrees must simply be stupid or at least incompetent. Well, if you can build a web framework that's as productive to use as rails but works 100x faster, you will be a millionaire practically overnight. Have at it. Forgive me if I don't hold my breath.

Oh but it really is adding frilly bits to data.

Have a good long hard look at the data flow of your web app. Take for example a CMS: A user comes along, the web page is generated, The data comes from a database, its encapsulated in HTML/JSON/x and then sent to the client. (either all at once or chopped up into bits and done asynchronously). If you're advanced, you might pull data from an API (still repacking data in a veneer of *ML)

I have a case of "shits already done" syndrome. For 99% of companies any old web framework will do. Those 1% that need a bit extra, spend the man hours on engineering ways round the deficiencies in the framework they chose. Most of that effort goes into expanding the namespace of the database, Because IO is a pain at scale.

I work in VFX where efficiency and scale really matter. I look after 16k cores and ~5+pb of tier 1 disk storage. 1% increase of efficiency in CPU usage is worth heafty cash. I've seen many, many fads. (Map+reduce, rails, mongodb, couchdb) All of them are re-inventions of the wheel (Map+reduce was seen as a task dispatching system, Rails was supposedly going to replace QT, Mongo/couchdb was supposedly going to replace a posix file system, and postgres as well)

We're now back at the stage where Postgres +SQL is awesome, Filesystems are good at storing unstructured data, and pythonQT really isn't that scary after all.

Although node.js and callbacks are starting to become popular now.....

I think we are talking about different things here.

I'm well aware that 99% of companies could get by with any old web framework. Should they, though? Of course not. Since we're talking about rails, let's use that example. It's an extremely productive, flexible, capable rapid application environment. And yes, it's not a speed demon, especially when used naively, no-one denies that. So why do companies choose it?

It's not because they're stupid or inexperienced. It's because the tradeoffs of higher productivity are worth the penalty in execution speed. Slightly higher server costs are worth the massive boost in productivity compared to other options. Why do you think it's so popular? If you're going to reply "because people are cargo cult following lemmings", you are simply wrong. The tradeoffs make sense to them, that's why.

So you're a sysadmin. Linux uses python all over the place. Why is that? They could have used any old language. But they chose python, even though it's slower than C, because the tradeoffs - easy to read, write, maintain - are worth it.

VFX could not be a more different environment than rapid web application development. As you say, 1% improvement means big bucks. You know what 1% improvement gets you on the web? Diddly shit. And every few percent raw speed increase on the web, as you drop down into the less dynamic frameworks, costs you massively in dev time, maintainability, hireability. It is a balance people strike. If I could increase my team's productivity 10% at the cost of doubling our server expenses I would take that deal instantly.

I can't speak to the fads you mention, except I also generally disdain them. But progress does happen. Which is why the web sites for the films you help make are probably not just written in any old web framework, they're probably written in rails or node or what have you, because the tradeoffs are worth it.

Remember what site you're on. This is about startups, small companies trying to grow fast. It's not about hyper-optimised rendering farms. Different worlds, dude!

Anyway, I think you actually agree with me, and I with you, if we just step into each other's shoes for a second.

Although I don't know what's so hard about making a couple of animated videos. 99% of the time you can just use any old rendering framework. I mean you just ask for a frame, it generates it, and saves it to disk right? Dude, I could do that with povray in 1995 on a 386. Whyever would you need 16 thousand computers for that? And 5PB of RAID? I can store a whole movie in like 3GB and that's 1080P! Actually my imac can make pretty realistic graphics in games, maybe you should buy one of them? Shit's easy right? : P

so long as you had a 386 with the floating point unit. Ironically for a while the iMac was the only choice until the retina for decent graphics. (and CPU, still waiting for the fucking dustbin)

>It’s extremely difficult to hire top talent these days; good developers like to learn new things and to use new technologies.

I have to wonder how true this is. Someone who gets good at whatever their domain is is probably better at that domain than the leapfroggers.

A single tech stack can only teach me a limited amount. When I build something around an unknown-to-me language/framework/tool over a weekend, I usually walk away with new insights.

For example, playing with Meteor (js) was the first time I was able to understand the concept of pub/sub.

That knowledge can sometimes be brought back to other tools I use, sometimes not. When it cannot easily transfer, the new tool gains brownie points. Playing with the "flavor of the month" taught me something useful. If I stayed with the same established workflow I always use, then I wouldn't have had a reason to encounter pub/sub.

Sorry, but that's nonsense, go and try Haskell for a year and you will have barely scratched the surface.

Well maybe, but does this seriously matter?

That's a very bad thing to spread. Haskell might be deep, but after using it for a whole year in production, any developer worth his salt would know enough of it to build non trivial things.

This type of comments unfortunately keep developers away from Haskell. Haskell is no C++, you can actually learn it, and it is a very good language.

Barely scratched the surface of what the language can do or the possible insights available by using Haskell?

I ask because I have done exactly what I described above, both with Haskell and Haskell frameworks.

Pushing things to the client is easier when the client and server speak the same language.