A List of Useful Backend Architectures | Hacker News Reader

A List of Useful Backend Architectures

gist.github.com

98 pointsyarapavan25 comments

Completely dominated by open source projects.

This helps my theory that when you release open source code, you are improving the baseline technology for the entire world.

Yep.

And when companies release the products THEY use and find helpful, everyone improves it and releases it back.

Memcache springs to mind.

Not necessarily. I have tons of github repos where nobody contributes to. I don't see why it wouldn't be the same for a company. It depends if your project is useful to others and if you're able to market it properly.

Yes, Yes, Yes.

OSS needs marketing, and the big ones recently have almost all sprung from world famous companies (twitter bootstrap, facebook and phabricator etc etc). It will become harder and harder to get noticed, but this is truly a open competitive market, and the best becomes top dog in most cases.

CEOs who spout support for the Free Market - beware, this is what it looks like. :-)

There is an extreme case of the availability heuristic at work here. There are a ton of incredibly successful companies that use little or no OSS.

http://highscalability.com/ is also a great blog for those interested in scaling backend architectures, they have great use cases

This is like discussing the architecture of buildings by listing the principal building materials

It's at least a good jumping off point to read more about how to use the materials to build the type of thing you want. I haven't even heard of many of them, but I work in the .NET world these days.

That wouldn't be a bad thing, actually. The choice of building material has implications far beyond what you'd imagine.

Agreed. At the very least, it would be helpful to know the specific applications of the languages in projects that use more than one.

This was disappointing. I came here actually expecting to see something about architecture but ended up looking at product stacks.

having said that, I am somewhat glad to see django there. for all the inconsistencies and speed issues that have already mentioned about python and django, it is great to see that the framework still gives pretty good performance on large systems.

While these aren't so much architectures as they are a list of tools, it's fascinating to see which ones successful companies are using.

I like that Trello has such a short list and still runs incredibly fast with a high number of users.

You say that like picking a small number of decent building blocks and then putting your energy into building high-quality code rather than wrangling a circus of services is a bad way to build scalable systems.

architectures != stacks

Agreed. This is a list of buzzwords, not a list of architectures.

An architecture would be something like 'all major work done via job queues distributed across cheap machines, all pages statically built and pushed to CMS', or 'a dozen bare-metal machines with serious hardware running one-machine sharded app/cache/db slices and hot backups of each shard', fun stuff like that. relatively technology agnostic.

Agreed, but it's still a nice list with a lot of information.

Check out the first chapter of "The Architecture of Open Source Applications (Volume Two)":

"Scalable Web Architecture and Distributed Systems" http://www.aosabook.org/en/distsys.html

What's with "my-sql". Shouldn't it be "MySQL"?

A a lot of these start-up (except for Facebook maybe ) are using python in their stack, why is that ? all I hear about around me is "Java Java Java". What makes python better ? ( it is just a question ). Or is python just for mvp then the stack moves to Java ( like Twitter did with rails )?

I'd advise against Python. There are too many caveats in it for a language that's supposed not to surprise me.

My biggest gripe: it's easy to shoot yourself in the foot using Unicode. The system is cumbersome and the lack of typing makes it easy to slip a regular ASCII string somewhere, which will jump at your face when you expect it the least morphed into a stack trace.

The data structures' API is the worst and most inconsistent API I've ever seen. Why is 'dict' not 'Dict' ?

A comma ending a line actually makes a tuple, that's asking for trouble. I've witnessed more than one bug caused by this.

It is slow and resource hungry. Django is too[1]. GlobalInterpreterLock[2].

The package system is horrible, and pip won't save your day either. I love Clojure and Node.js just because of Leiningen/NPM. I couldn't use PyOpenGL in Windows from some unknown reason. I gave up!

Discourages functional programming, while encouraging ugly inline list comprehensions. The solution to the half-assed closures in Python2.7[3] was to introduce 'nonloncal'. Lists are mutable. Object Orientation feels clunky to me and using super is awkward.

"Batteries included" but core libraries won't take you too far. Unfortunately they're still bloating the language.

Concurrency is hard. Twisted solves it a bit but... meh...

What I like:

- Proper nested comprehensions (CoffeeScript completely blew this one...)

- I like the minimal syntax, it helps me visualize the program better.

- PyLint/PyFlakes highlights lines that don't conform to PEP8.

- Generators.

[1] http://www.techempower.com/benchmarks/#section=data-r5

[2] http://ynniv.com/blog/2007/08/closures-in-python.html (section Caveats)

[3] http://wiki.python.org/moin/GlobalInterpreterLock

What would you use instead of Python? It should be roughly comparable in terms of having a wide array of well-used libraries, and of course a good REPL.

I've been using Python for a couple years and it's mostly joyful. It is unfortunate that not even the various containers within "collections" use a uniform naming convention, I'll give you that. And some of the scoping stuff is weird (nonlocal which nobody really uses? and seemingly-but-not constant default function arguments). But Twisted is a pretty good evented programming system, and NumPy plus multiprocessing let you process a lot of data efficiently, GIL or no. I cannot agree that "the core libraries won't take you too far"--they've taken me farther than the comparable bits in any other language.

I used to be a huge python fan but after a couple of years of scala I don't want to go back. The syntax can be almost as lightweight, the typing is just a huge help as your project gets bigger, and the whole language just feels a little more consistent, with fewer weird corner cases like the default arguments thing you mention.

Supposedly many python programmers are switching to Go (devs were suprised too).

There's nothing in your list that makes Python less suitable for web development than [insert favorite language]. Many of the issues you describe (I agree with most but not all) are either irrelevant for web application development (e.g. the GIL, I/O bound code such as web applications do not suffer from it at all, especially not when using something like gevent), easily avoided or worked around (unicode strings: either use Python 3, or a web framework that defaults to always unicode + do static source analysis for non-unicode string literals), already have pretty good solutions (package management: always use virtualenv and pip), or a matter of style/taste (dict vs Dict, functional aspects, etc).

I could make my own list of reasons why I think Java sucks for web development, but someone who's proficient in Java web application development will disagree with those just the same. Every language has its faults, and Python is no exception. Personally I simply hate writing Java code, of all the languages I've used, it's by far the least enjoyable to write and maintain.

nah, ruby is pretty much perfect hth

(the slowness of the interpreter allows you time to appreciate its beauty)

> A comma ending a line actually makes a tuple, that's asking for trouble. I've witnessed more than one bug caused by this.

Spurious symbols at the end of a line is really a complaint? Have you ever forgotten a semi-colon in c/c++ code?

Reply on news.ycombinator.com