Comment by KaiserPro | Hacker News Reader

Comment by KaiserPro | Hacker News Reader

Parent Full thread

Oh but it really is adding frilly bits to data.

Have a good long hard look at the data flow of your web app. Take for example a CMS: A user comes along, the web page is generated, The data comes from a database, its encapsulated in HTML/JSON/x and then sent to the client. (either all at once or chopped up into bits and done asynchronously). If you're advanced, you might pull data from an API (still repacking data in a veneer of *ML)

I have a case of "shits already done" syndrome. For 99% of companies any old web framework will do. Those 1% that need a bit extra, spend the man hours on engineering ways round the deficiencies in the framework they chose. Most of that effort goes into expanding the namespace of the database, Because IO is a pain at scale.

I work in VFX where efficiency and scale really matter. I look after 16k cores and ~5+pb of tier 1 disk storage. 1% increase of efficiency in CPU usage is worth heafty cash. I've seen many, many fads. (Map+reduce, rails, mongodb, couchdb) All of them are re-inventions of the wheel (Map+reduce was seen as a task dispatching system, Rails was supposedly going to replace QT, Mongo/couchdb was supposedly going to replace a posix file system, and postgres as well)

We're now back at the stage where Postgres +SQL is awesome, Filesystems are good at storing unstructured data, and pythonQT really isn't that scary after all.

Although node.js and callbacks are starting to become popular now.....

I think we are talking about different things here.

I'm well aware that 99% of companies could get by with any old web framework. Should they, though? Of course not. Since we're talking about rails, let's use that example. It's an extremely productive, flexible, capable rapid application environment. And yes, it's not a speed demon, especially when used naively, no-one denies that. So why do companies choose it?

It's not because they're stupid or inexperienced. It's because the tradeoffs of higher productivity are worth the penalty in execution speed. Slightly higher server costs are worth the massive boost in productivity compared to other options. Why do you think it's so popular? If you're going to reply "because people are cargo cult following lemmings", you are simply wrong. The tradeoffs make sense to them, that's why.

So you're a sysadmin. Linux uses python all over the place. Why is that? They could have used any old language. But they chose python, even though it's slower than C, because the tradeoffs - easy to read, write, maintain - are worth it.

VFX could not be a more different environment than rapid web application development. As you say, 1% improvement means big bucks. You know what 1% improvement gets you on the web? Diddly shit. And every few percent raw speed increase on the web, as you drop down into the less dynamic frameworks, costs you massively in dev time, maintainability, hireability. It is a balance people strike. If I could increase my team's productivity 10% at the cost of doubling our server expenses I would take that deal instantly.

I can't speak to the fads you mention, except I also generally disdain them. But progress does happen. Which is why the web sites for the films you help make are probably not just written in any old web framework, they're probably written in rails or node or what have you, because the tradeoffs are worth it.

Remember what site you're on. This is about startups, small companies trying to grow fast. It's not about hyper-optimised rendering farms. Different worlds, dude!

Anyway, I think you actually agree with me, and I with you, if we just step into each other's shoes for a second.

Although I don't know what's so hard about making a couple of animated videos. 99% of the time you can just use any old rendering framework. I mean you just ask for a frame, it generates it, and saves it to disk right? Dude, I could do that with povray in 1995 on a 386. Whyever would you need 16 thousand computers for that? And 5PB of RAID? I can store a whole movie in like 3GB and that's 1080P! Actually my imac can make pretty realistic graphics in games, maybe you should buy one of them? Shit's easy right? : P

so long as you had a 386 with the floating point unit. Ironically for a while the iMac was the only choice until the retina for decent graphics. (and CPU, still waiting for the fucking dustbin)

Reply on news.ycombinator.com