Anyone got any insight into why python was chosen?.
Initial release of NodeJS was May 27, 2009; 14 years ago.
Swartz was involved in the development of Reddit until he departed from the company in 2007.
And we know that parts of the Python code for Reddit was written by Swartz.
This is about the “scripting niche” I was talking about: in 2008 every Linux distro included Python and lots of build scripts were written in Python 2, but Python for the back-end was really rare back then.
Python was definitely around, had a very strong community, and a number of very significant programs written in it. (Among them: the first versions of BitTorrent and Google.) However, if you were a random dev applying for a random job, you would most likely have to work in C++, Java, or C#. It was relatively difficult getting a job where you would write the majority of your code in Python. Certainly they existed, and many jobs would have you write small one-off scripts in Python, but the actual core product would usually be one of Java/C#/C++.
Remember that in 2009, Google had 20K employees while IBM (which was a major booster of Java) had 400K. And at Google, Python wasn't really allowed for major projects (Search had been rewritten from Python to C++ when it got big in 2000), so you'd be using Java or C++ anyway. Data science wasn't really a thing, and if you did machine learning you used frameworks like Weka or home-grown stuff.
The internal tooling we built was mostly in Python.
Baby boomers always tell me about seeing The Doors live in dive bar in NYC. I can say I used Reddit when it was written in Lisp. :-)
Even years after, when I had stopped posting on those PHP powered smaller forums I resisted creating a reddit account just because it was "reddit".
I would say, the smaller forums sites were a more tightly knitted group that I attended several meetups with. Reddit just never felt the same.
https://web.archive.org/web/20060206185841/http://reddit.com...
Why they moved away from CL:
"If Lisp is so great, why did we stop using it? One of the biggest issues was the lack of widely used and tested libraries. Sure, there is a CL library for basically any task, but there is rarely more than one, and often the libraries are not widely used or well documented. Since we're building a site largely by standing on the shoulders of others, this made things a little tougher. There just aren't as many shoulders on which to stand."
And why Python was chosen:
"So why Python?
We were already familiar with Python. It's fast, development in Python is fast, and the code is clear. In most cases, the Lisp code translated very easily into Python. Lots of people have written web applications in Python, and there's plenty of code from which to learn. It's been fun so far, so we'll see where it takes us."
So, basically, they were happy with CL except for lack of libraries, and because they also knew Python already, and they knew there were more libraries for Python, they picked it. Simple as that.
But I contributed a tiny bit of code to reddit back when it was open source and chatted with reddit engineers on IRC, so I do know the official justification (as of 2010ish) for reddit sticking with python:
Websites are mostly IO bound anyway. The hot parts like markdown formatting and templating are already c modules. So the performances gain for rewriting in a compiled language aren't large enough to justify it.
On top of it being slow and brittle, dependencies break, needlessly destabilizing stuff.
Worse, the python team ends up having the hardest job, due to self-inflicted problems, so it either ends up with junior devs that don’t know better, or bitter senior devs that could be 10x more productive doing something else.
As always, it varies from company to company, but this is what I saw on four teams out of four at multiple companies.
You can abuse dependencies in every language. This doesn't sound like a Python problem but a bad tech management problem (i.e. who signed off on allowing 'randomguy69/left-pad' as a dependency).
done a few services and CLIs and never had updating deps break anything unless you count newer linters complain about new things.
so i would be surprised if a server written in python could saturate the network, for a reddit-style workload, which i imagine would be similar.
any1 have relecent experiences to share?
With the "new" reddit they replaced most of the frontend with javascript and it really shows. That's not to say I like sites that use too much javascript but python is slow enough just parsing and filling html templates, seemingly.
Visual noise is like inflation. A constant tax on what you do. But you barely notice it. But it accumulates because it is constantly leading to slightly worse and more complex code. Since it is harder to focus on a noisy something in front of you, you make slightly worse decisions. Over time this accumulates. And similar to how Warren Buffet noticed that a Dollar of his youth is not even worth a penny these days, over time, exponentially more effort is needed to develop the more and more complex system.
So over a longer timeframe, a visually more elegant language wins by a long shot.
Python is the visually most elegant language.
I just took a look at the 4 latest Show HNs which are written in Python. None of them uses types.
Usually someone will pop up showing crazy meta programming template code and use it a straw man to declare cpp=bad.
And I say that as someone doing C++ for a living, and occasionally engaging in said crazy template metaprogramming stuff. Occasionally, because people tend to frown at it during code review; apparently, metaprogramming is reserved only to library authors...
For a company complaining about infrastructure costs for a product that has been cloned many times over, and which is large enough to hire offshore devs, the cost equation favours rewriting in something like Go, Rust or even Java. These are quite tailored towards Reddit’s current use case and many companies including Twitter did that rewrite.
Making better decisions on a daily basis compounds. You evolve towards a better architecture. Which can make the code multiple orders of magnitude faster.
One of many examples: In a complex system, developers often have a blurry vision of what can be cached, in which situations it can be cached, and how cache invalidation should be done. While in a more elegant system, the borders between different approaches to caching are clearer. Leading to the more elegant system being multiple orders of magnitude faster as it can leverage the different caching layers (in memory cache, on disk cache, the CDN ...) more effectively.
Having program login buried in a nauseating amount of boilerplate doesn’t feel very “visually elegant” :-P
The performance critical part is the database, and that at least used to be written in Java (cassandra db).
Writing the bit of glue code between your high performance database and your fronted in cpp would introduce all kinds of really bad potential bugs, scary remote code execution bugs.
CPU usage of that glue code might take up 10% of your hardware budget worst case, even with a "slow" language. It's just not a concern.
Well, given complaints how much 3rd party apps cost them, they’d better off optimize their crap first.