Yes, Python Is Slow, and I Don’t Care
hackernoon.com
hackernoon.com
The author presents some flimsy "study" about coding string processing algorithms with a straight face, and basically loses all credibility. What about lifetime productivity of a codebase? Speed of refactoring, onboarding, testing etc..
Why don't you just be honest and say you don't care Python is slow because you like using it?
Performance doesn't matter? Of course it does. On the frontend, bad performance has been proven by many eCommerce companies to lead to lost business. On backend data pipelines, bad performance kills your flexibility. The longer it takes you to process N, the less times you can do it. The more times you can do it, the more times you can get it wrong.
Also who says that time-to-market is everything? Tech history has shown that product quality/brand marketing win out in the long term. There are tons of forgotten shitty products that were first. Being first is over-rated.
The idea that one chemical could be proven to be more effective than another is an equivalent problem. History, application, duration, there's a permutation of issues under that umbrella, as well. The problem is that nobody wants to argue about it very much. The Quorum project tried, to a small degree.
> Why don't you just be honest and say you don't care Python is slow because you like using it?
It's that and that you have an existing toolchain/codebase you don't want to rewrite in a potentially SLOWER way, because you don't know another language well enough. Python devs rather "solve real problems" than worry about a class of problems that their tool is inadequate for. There are always new trivial problems, where performance doesn't matter.
I disagree. In the extreme, take python on the one hand and a variant of brainfuck that requires the hash (could be as simple as CRC-32 or as hard as SHA-256) of every source file to be zero on the other.
I don’t think you have to go that far to get provable differences, though. Some features that, I think, can fairly easily be proven to make programmers more productive in large parts of the space of programming tasks:
- assembler macros
- variable names that can have more than two characters
- direct support for floating point
- support for complex numbers
- local variables
- structures
- support for serialization and deserialization. Compare the way one can make simple structures serializable using Swift’s Codable protocol to whatever is ‘best’ for C (probably something macro-based)
- generics
- support for Unicode strings
- iterators
Of course, “python vs Java” or “python vs Perl” are much closer races where I expect measuring differences is way harder and results will depend on problem domain, experience of the subjects of the study, number of good and bad answers on StackOverflow, etc, but even for ‘normal’ languages, I expect some conclusions to be possible, given enough effort is spent. Python 1.0 didn’t have complex numbers, for example, so if your program needed them, python 1.4 was a huge improvement.
I doubt evaluating productivity of languages is worthwhile, though.
I’m practically forced to use python for quant finance at my job because of the network effect: numpy, pandas, cvxpy, etc.
But the language itself is total garbage compared to, say, Scala or OCaml. And both of those languages are also much faster.
Lambdas suck, collection types are totally inconsistent (list, dict, etc), too many magic keywords and concepts. Trying to program using a FP style is almost impossible. Also no macros, no traits, no real classes (duck typing is garbage and lazy).
You know a language is garbage when the most popular libraries aren’t even written in the language!
Like, I'm partial to Haskell, but if I'm actually trying to get work done, I reach for a language with a robust ecosystem. These days I reach for Typescript, but before then, python was my go-to since it's way nicer to program with than java/c++/c/javascript.
Unfortunately for me, those are all things I do at my job. Outside of these domains, JVM libraries are much better.
No, it's not. When I switched from Python to OCaml it felt like blessing, since the quality of the libraries felt much higher in OCaml. There are also the projects that you have to build from the ground up without libraries (any mission critical software), and in such cases you don't care about libraries at all, you care about the merits of the language solely.
Python is the BASIC of the XXI century, and it's fine in that role, but not as a general purpose language.
How about compared to a functional language like Haskell, OCaml, or F#?
I agree that we should choose productivity over performance when possible. However, Java, for example, is extremely verbose.
My impression is that people who use functional languages are quite productive.
Plus, they need fewer unit tests.
The same researcher must write a study of the median temperature at the hospital.
That said, I have to wonder if these dynamic scripting languages are truly enabling productivity or if it just feels like that because you can write it and try many things quickly. It reminds me of being such in traffic on the highway - sometimes I get off and take back roads even if my GPS tells me it will take longer, because it is more pleasant to be moving than feeling stuck.
That said, I think the biggest benefit of dynamic languages is that they are so easy to refactor. You don't have as rigid a structure, so you can try easily hack together some idea that, done "properly", would require a big refactor. Despite being functional, Haskell's strong type system makes refactoring a huge pain. If course the same things enabling this quick refactoring in the dynamic languages works against you when it comes to speed and reliability.
1) its usually founded on a very good understanding of the problem, and the abstractions inherent in the problem. If he can re-use the abstractions he has used before he does, to great effect in speed-to-solution
2) it is very usually correct. it is remarkably bug free because the Haskell typing removes a huge amount of ambiguity about the problem going in
3) he has absolutely no problem explaining what it does 3 months or more later.
I value these behaviours immensely. My own Python is usually buggy, often re-produces techniques I did a week ago, badly, and I can't explain why I used that lambda/map function.
Some stuff connecting type: value record structures into JSON.
We work a lot on ill-formed requests to refine data previously worked on with new questions. We work a lot on small distributed systems exploring eventual consistency and event sourcing.
We work with quite large logfiles collected asynchronously, with poor clock alignment. So data buffering and sorting is not uncommon.
Slower to write (and get right) -> fewer bugs to deal with later
Pick your poison
Having your company survive (because you got something faster to market), and having bugs to deal with later, is a so-called "nice problem to have".
The bug-less software of a dead company does nobody good.
(It depresses me that I have to write a defense of good engineering on Hacker news. The value of doing things correctly should be a tacit core principle of any engineer, to be broken only under circumstances of exceptional duress.)
In those places, they also have other kinds of compliance requirements for the code -- which makes sense: I wouldn't leave the airline software PMs to chose between the above two options.
(Typical SV answer: you're only thinking of the short term because you're trying to get investment and you plan to exit stage left with a fat packet long before the technical debt rears its ugly head, leaving somebody else to clean up the mess. But this is not how most engineering is done in the real world.)
Even in those that aren't, the managerial "finish the job by next Thursday or you're fired" is equally good motivator to not "save time and do it right the first time".
And don't get me started on the "specialist" consultants, whose job is to push some monstrosity, like IBM's stack, or SAP, or whatever, until it kinda fits the problem domain, and in the process sell even more crap from their vendor.
Some fun stuff (haven't checked them all, but most stuff I've read from WTF across the years was hilarious, if not true to life):
https://thedailywtf.com/articles/Consulting_The_Consultants
https://thedailywtf.com/articles/Invasion-of-the-Consultants
https://thedailywtf.com/articles/consultant-designed-success
(And inversely, there are companies whose culture is all about good code and craftsmanship. But they usually don't get very far: if they don't fold, they're still niche shops. Perhaps Viaweb was one such).
The point was that while for consultancy projects, customers do refuse to pay for what is considered a bad piece of work, for other kinds of software delivery, with exception of app stores, that never happens.
It is very interesting experience to take part in a customer meeting, which isn't very pleased how things are going, regardless of the outcome.
I also have my share of being part of such teams, similar to your examples, yet that doesn't mean one should stop pushing for quality as much quixotic endeavour as it might seem.
Software is infinitely malleable and can be used under all kinds of environments (e.g. hardware configurations, OS versions, etc).
It's more like a novel than a microwave. When you buy a novel you don't get your money back when it doesn't perform well, or puts you to sleep in the 10th page.
It is just that a large majority of population isn't aware of their consumer rights, or just doesn't want to bother.
Does it surprise you, though, on Hacker news?
No need to sacrifice performance for productivity.
[1] - Yeah, I know it is dead now, just wanted to mention it anyway.
Also there's a big difference between throughput and latency. Networking might introduce a longer delay than a CPU call, but if each one of those calls does significantly more work, the overall performance of the system can increase.
If it's a one off or otherwise rarely executed, yeah, it doesn't matter much. But it starts to matter when it's constantly running on a significant number of systems.
Widely distributed frequently running slow code can waste a lot of energy. Cell phones and laptops heat up and have a shorter battery life. Even desktops require more power.
On the global scale, a large number of such systems can waste several nuclear reactors worth of electricity.
10 times slower code might require 10 times the number of servers. Maybe it doesn't matter much when it's just the difference between one and 10 servers. Or perhaps you don't even consume 10% of a single server. But there's a point when the hardware and power costs start to dominate.
The scale matters.
Python isn't slow. Programming languages aren't slow or fast but interpreters are.
cpython is slow. PyPy interpreter is reasonably faster, upto 8 times.
Recently pypy 7.0 released which brought more hope.