Why Python keeps growing, explained
github.blog
github.blog
Most of these applied fields are using Python and R as no more than data gathering tools and fancy calculators. something for which the benefits of other languages are just not justified.
The absolute beauty of Python for what I do is that I can write code and hand it off to a first year with a semester of coding experience. Even if they couldn’t write it themselves, they can still understand what it does after a bit of study. Additionally, I can hand it off to 75 year old professors who still sends Fax memos to the federal reserve and they’ll achieve a degree of comprehension.
For these reasons, Python, although not perfect, has been so incredibly useful.
And I wouldn't be surprised if a large number of python programmers would say they're not programming, it's just scripting.
It’s still my choice for quick and non-graphical analysis when I’m on a remote.
Today honestly most tools are pretty capable, pandas is a great choice and if you have really high volumes of data you might try koalas (spark) or polars.
Honestly the biggest design considerations for data science today are things things external to your project: what do you and others on your team know, what tools does your company already have setup, what volume of data are you processing, what are your SLAs, who or what else needs to run this script/workflow, what softwares do you need to integrate with, how often does it need to be processed, how are you going to assure the quality of your data and what tools are you using for reporting?
I tend to use pandas and SQLite for most use cases cause I can cook up a script in 2 hours and be done, I just code it interactively in a notebook and most people are able to work on a pandas or SQLite script productively if it needs to be maintained even if they don't know python. If its a large volume of data or a rapid schedule (minutes, seconds) or tight SLAs on quality or processing time, then I start to consider whether pyspark, Apache beam, dask or bigquery might be a good fit.
So it really just depends but for most people who are processing < 100 GB on a 1+ day schedule or ad hoc I would recommend just using pandas or tidyverse in R and getting really good at writing those scripts fast. Today you’ll get the most mileage out of those two tools.
They aren't technically "spaghetti", but they are technically impenetrable.
I argue that one-liners like that aren't good for anybody, dev or otherwise.
When writing code that will be used by a particular sort of user base, the code should be written in whatever way best suits that user base. If your users are academics, researchers, journalists, etc. -- yes, avoid anything with complex or obscure semantics like perl or bash.
But if your code is going to be used by programmers or people who are already comfortable with perl/bash/whatever, those tools may be just the ticket.
If you use the python repl a lot and haven't heard of it, ptpython is worth checking out as a repl replacement. I find it to be much more ergonomic.
No need to fire up a python repl.
You’ll find everyone from philosophy PhDs to Biologists to Journalists who use pandas because its so easy to learn it and work with it. It’s amazing how you can become productive in python/pandas without any experience or even basic understanding of programming because of how accessible jupyter, colab and blogs/docs on pandas are.
The other thing people don’t talk about is that a lot of these organizations can hire a CS student part time or a full time software engineer/data engineer/data scientist who can optimize their scripts once they are written. Pretty much any software engineer can read and debug python code without needing to learn python. So for example, I know some engineers working in genomics who have turned biologist-written scripts that take several days to run in python into scripts that take hours or minutes to run by doing basic optimizations like removing quadratic algorithms from the script or applying pyspark or dask to add parallelism.
The fact that python can be used as a bridge between technical and non-technical people is amazing and I think it has provided a better bridge between these groups than SQL was ever able to provide.
Nobody cares about this that much. Even a straight up software developer in python doesn't care. The interpreter is so slow that most optimization tricks are irrelevant to the overall bottleneck. Really optimizing python involves the FFI and using C or C++, which is a whole different ball game.
For the average python developer (not a data scientist) most frameworks have already done this for you.
1) Languages like Python / R / Julia / etc. + SQL
2) PowerBI, Tableau, or similar tools
3) Excel
The number of users of those tools will be the inverse, with Excel being number 1.
If you're competent using the "stack" above, you could probably work as an analyst anywhere - given that you can pick up domain knowledge.
This piqued my curiosity. I've worked with Python on and off for the last ~20 years, and while I'm not a fanboy or apologist, and use other tools when appropriate, there's also a reason it remains in my toolbox and sees regular use while many other tools have come/gone/been replaced by something better.
Can you share an example scenario where it's a Monkey's Paw? My suspicion is that this is more of an org issue than a tech issue?
People work around these issues with various kludges like virtual environments, Docker (just ship the whole system!), and half a dozen different package managers, each with their own manifest format. But this is a problem that simply doesn't exist in Go, JavaScript, Rust, and others.
For code that never needs anything except the standard library, or for a script that never needs to be maintained or run on a different machine, Python is fine. Maybe even nice. But I've watched my coworkers waste so many hundreds of developer-hours just trying to wrangle their Python services into running locally, managing virtual environments, keeping them from trampling on each other's global dependencies, following setup docs that don't work consistently, and fixing deployments that fail every other week because the house is built on sand.
Virtualenvs, and requirements are a thing in Python for ages.
I’ve used tons of languages and while not the best, Python dependency management and project isolation is decent. IMO certainly better than JavaScript.
In contrast to Javascript/Node.js/NPM/Yarn/whatever-you-want-to-call-server-js, which maintains a local folder with dependencies for your project, instead of installing everything globally by default.
Heck, a virtual env is literally a bundled python version with the path variables overriden so that the global folder is actually a project folder, basically tricking Python into doing things The Correct Way.
fp languages like
ocaml
haskell
racket
common lisp
erlang
elixir
or rust/golang
but even if the code is single threaded and not hampered by GIL limitations python tends to be super slow imho; also debugging dynamic python and imperative stateful python after a certain code base size >10k LOC gets extremely painful
When we really want or need performance we rewrite the slow part in C++ and use pybind to call into it. For all the real implementations that run in a soft real time system, everything is done in C++ or C depending on the ecosystem.
for any meaningful scale you are better served by basic FP hygiene as evidenced in
haskell
elixir
CL/racket
or rust/golang
(loop for x across numbers
when (evenp x)
do (setf result (+ result x)))
I feel that it has recently - like many really mature platforms - become very much like the elephant from that old apocryphal story [0]. It is being used for many different purposes, with very different requirements and needs, with users being so focused on their own use that anything outside that is considered "bloat" and "waste".
By the time a python programmer has «graduated» to learning a second language, exponential growth has created a bunch of new python programmers, most of which don’t consider themselves programmers.
There are more non-programmers in this world, and they don’t care - or know about - concurrency, memory efficiency, L2 cache misses due to pointer chasing. These people all use python. This seems to be a perspective missing from most hackernews discussions, where people work on high performance Big corp big data web scale systems.
If I write something in C++ that does a task in 1 second and it takes me 2 days to write, and I write the same thing in Python that takes 2 seconds but I can write it in 1 day, the 1 day of extra dev time might just pay for throwing a more high performance server against it and calling it a day. And then I don't even take the fact that a lot of applications are mostly waiting for database queries into consideration, nor maintainability of the code and the fact that high performance servers get cheaper over time.
If you work at some big corp where this would mean thousands of high performance servers that's simply not worth it, but in small/medium sized companies it usually is.
I have code running that reads ~20 bytes, checks the internal status on an hashmap and flips a bit.
Would it be faster in C? Of course.
Would it have taken me much longer to write to achieve absolutely no benefit? Yes.
This is the first line in most scientific code:
import numpyWhat slows Python down is generally the "everything is an object" attitude of the interpreter. I.e. you call a function, the interpreter has to first create an object of the thing you're calling.
In C++, due to zero-cost abstractions, this usually just boils down to a CALL instruction preceded by a bunch of PUSH instructions in assembly, based on the number of parameters (and call convention). This is of course a lot faster than running through the abstractions of creating some Python object.
Nah, it’s the interpreter itself. Due to it not having JIT compilation there is a very high ceiling it can not even in theory surpass (as opposed to things like pypy, or graal python).
Python is primarily slowed down because of the fact that each attribute and method access results in multiple CALL instructions since it's dictionaries and magic methods all the way down.
How so? Python is dynamically typed after all and even type annotations are merely bolted on – they don't tell you anything about the "actual" type of an object, they merely restrict your view on that object (i.e. what operations you can do on the variable without causing a type error). For instance, if you add additional properties to an object of type A via monkey-patching, you can still pass it around as object of type A.
Also, if you are interested, “shapes” are the primitives of both Javascript and python jit compilers instead of regular types.
Commonly it creates the .pyc files, so it doesn't really re-parse your code as a string every time. But it does check the file's dates to make sure that the .pyc file is up to date.
On debian (and I guess most distributions) the .pyc files get created when you install the package, because generally they go in /usr and that's only writeable by root.
It does include the full parser in the runtime, but I'd expect most code to not be re-parsed entirely at every start.
The import thing is really slow anyway. People writing command lines have to defer imports to avoid huge startup times to load libraries that are perhaps needed just by some functions that might not even be used in that particular run.
That doesn’t really take any significant time though on modern processors.
And OCaml (which offers a productivity comparable to Python) is sensibly slower than Rust or C++.
But yes python is slow.
However I've seen good python code be faster than bad C code.
I was talking specifically of pure python code (except the python's standard library itself, where it really is unavoidable).
But sometimes, you do end up writing that compute heavy piece of code. At that stage, you have to learn how to write your own native library :)
Speaking of which, I've written some Python modules in Rust using PyO3, its' a very agreeable experience.
Code is "compute heavy" (could equally be memory heavy or IOPs heavy) if it's deployed into many servers or "the cloud" and many instances of it are running serving a lot of requests to a lot of users.
Then the finance people start to notice how much you are paying for those servers and suddenly serving the same number of users with less hardware becomes very significant for the company's bottom line.
The other big one is reducing notable latency for users of your software.
With Python I can write and test on a Mac or Windows and easily deploy on Linux. I can iterate quickly and if I really need "performance" I can throw bigger or more VPSes at the problem with little extra cognitive load.
I do not have anywhere near the same flexibility and low cognitive load with C++. The better performance is nice but for almost everything I do day to day completely unnecessary and not worth the effort. My case isn't all cases, C++ (or whatever compiled language you pick) will be a win for some people but not for me.
E.g. people who once wrote "robust" code in Rust but were "outcompeted" left and right by coworkers who churn out shiny new things at 10x the speed.
With Python I can get things up and going very quickly with little boilerplate, but I find that I'm often stumbling on edge cases that I have to debug after the fact and that these instances necessarily happen exactly when I'm focused on another task. I also find that packaging for other users is a major headache.
With Rust, the development time is much higher for me, but I appreciate being able to use the type-system to enforce business logic and therefore find that I rarely have to return to debug some issue once I have it going.
It's a tough trade-off for me, because I appreciate the velocity of Python, but Rust likely saves me more time overall.
Sounds like a classic web-dev perspective, my customers hate when we ship broken tools because it ruins their work, new feature velocity be dammned. We love our borrow checker because initially you run at 0.5x velocity but post-25kSLOC you get to run at 2x velocity, which continues to mystify managers worldwide.
People use Python in financial applications, Data Engineering and AI/ML pipelines, infrastructure software etc and the 10x speed can be real.
With Python, testing, good hygiene and a bit of luck you can write core that is maybe 99% reliable. It is very, very hard to get to (100-eps)% for eps < 0.1% or so. Rust seems better suited to that.
Anything else, especially if there isn't a huge premium on speed, meh - Python is almost always sufficient, and not in the way.
I don't need rust specifically. It's just that its memory and thread management really help me to continue what I do in python: focusing on my core business instead of technical stuff.
The less I code the better I feel :-)
Is a coping canard invented by programmers who can't into more powerful programming languages.
By far the slowest language for developing is PHP, it's even worse than plain C in that regard.
I'm sure Rust was the wrong tool for the job in your case but I find this type of get shit done argument unpersuasive in general. It overestimates the value of short-term delivery and underestimates how quickly an investment in doing things "the right way" pays off.
Business owners definitely prefer less bugs, less customer complaints, less support burden, less outages, less headaches. Corner cutting doesn't make economic sense for most businesses and good engineering leadership doesn't have much trouble communicating this up the chain. The only environment where I've seen corner cutting make business sense is turd polishing agencies whose business model involves dumping their mistakes on their clients and running away so the next guy can take the blame.
What business owners actually don't like at all is how long is takes traditional software development to actually solve problems - which then don't really fit the business after wasting a few years of ressources... and the dumping and running away is worse in Java and other compiled software. With Python you can at least read the source in production if the team ran away...
Java (and dotnet, the two big "VM" languages) is somewhat of a strange example for that; JVM bytecode is surprisingly stable and reverse engineering is reasonably easy unless the code was purposely obfuscated - a bad sign on any language anyways.
> I'm sure Rust was the wrong tool for the job in your case but I find this type of get shit done argument unpersuasive in general.
Unless you're working on a fire-and-forget project with a tiny time horizon get shit done arguments are blatantly short-termist.
If you're dealing in areas with short time limits then Python is great, because you can't sell a ticket for a ship that has sailed.
And I've seen "the right way" which, again, depending on the business may result in a well designed product that is not what's actually needed (because people are really bad at defining what they want)
What's brilliant with Python compared to other hacky solutions that it does support test, type hints, version control and other things. It just doesn't force you to work that way. But if you want to write stable, maintainable code, you can do it.
That means you can write your code without types and add them later. Or add tests later once your prototype was been accepted. Or whenever something goes wrong in production, fix it and then write a test against that.
Oh and I totally agree you should certainly try to "do things the right way", if the business allows it.
The only thing that can compete with it for productivity in the science space is R.
For an early stage start up this is almost the only relevant factor for success.
> underestimates how quickly an investment in doing things "the right way" pays off.
What time horizon should a startup optimize delivery for? Minutes, hours, days, weeks? Say you're a startup dev in a maximalist "get shit done now" mindset so you're skipping types, tests, any forethought or planning so you can get the feature of the week done as fast as possible. This makes you faster for one week but slower the week after, and the week after, and the week after that.
Say a seed stage startup aims for 12 months runway to achieve some key outcomes. That's still a marathon. It still doesn't make sense to sprint the first 200 meters.
It's just an application of the Pareto principle. That last 20% of work to make perfect software costs a lot of time. Customers (and by extension, management) do not care how pretty your code is, how perfect your test coverage is (unless your manager is a former developer, then they might have more of an opinion), they care most that you ship it. Bugs are a minor irritation compared to sitting around waiting for functionality they need, as long as you're responsive in fixing the bugs that do come up.
Of course there are at least three dimensions to discuss here: internal quality, external quality and product/feature fit. Lower quality internal code eventually leads to slower future development and higher turnover as no one wants to work with the crappy code base. Lower external quality (i.e. bugs) can lead to customers not liking your product. Interestingly the relationship between internal and external quality is not as direct as one might think. Getting features out the door more quickly (at the expense of other things) can help with product fit. Essentially, like most things, this is an ongoing optimization problem and different approaches are appropriate for different problem domains.
I am tired of having to refactor shiny new things churned out at 10x the speed and that keep breaking in production. These days, if given a choice, I prefer writing them in Rust code, spending more time writing and less time refactoring everything as soon as it breaks or needs to scale.
That's when rewriting the hot path in Go or Rust or Java or C or C++, can pay off and make those skills very valuable to the company. Making contributions to databases, operating systems, queueing systems, interpreters, Kubernetes etc. also fall into that category.
But yeah if you are churning out a MVP for a new business, yeah starting with Python or Ruby or Javascript is a better bet.
(Erlang/Elixir is also an interesting point in the design space, as it's very high level and concise, but also scales better than anything else, although not especially efficient for code executing serially. And Julia offers the concision of Python with much higher performance for numerical computing.)
(Fwiw I use all three, Python professionally.)
Also if I (a programmer) want to write really really fast code I'm probably reaching for tools like tensorflow, numpy, or jax. So there's not much incentive for me to switch to a more efficient language when as near as I can tell the best tooling for dealing with SIMD or weird gpu bullshit seems to be being created for python developers. If you want to write fast code do it in c/rust/whatever, if you want to write really fast code do it in python with <some-ML-library>.
For a very specific definition of the word "fast" at least.
Very limited view which ignores way too many areas.
Really, for almost any X, somebody somewhere has to do X with strict performance requirements (or at very large scale, so better perf == savings)
Most of these python libraries are only fast for relatively large and relatively standard operations in the first place. If you have a lot of small/weird computations, they come with a ton of overhead. I've personally had to write my own fast linear algebra libraries since our hot loop was a sort of modified tropical algebra once.
Rather unfortunately, my current bugbear is that Pytorch is... slow. On the CPU. One of the most common suggestions for people who want stable diffusion to be faster is, wait for it, "Try getting a recent Intel CPU, you'll see a real uplift in performance".
This despite the system only keeping a single CPU core busy. Of course, that's all you can do in Python most of the time.
(You can also use larger batch sizes. But that only partially papers over the issue, and also it uses more GPU memory.)
Whereas before you could get smarter programmers using Python, now because of the exponential growth of Python, the median Python programmer is likely someone with little or no software engineering or computer architecture background who is basically just gluing together a lot of libraries.
The libraries are the killer feature for me.
Anything that doesn't require high performance that is. Is there any 3D game engine for python yet? I guess Godot has gdscript which is 90% python by syntax, but that doesn't quite count I think.
If people write program logic in python it will run at python speeds. Otherwise you're not really writing python, like nobody says some linux native program is bash because it happens to be launched from a bash script.
But that's exactly the strength of Python: it's an interface language. It's meant to make pretty sophisticated things like CUDA easy for everyone.
But that's how every scripting language obtains good-not-just-decent performance. A strong culture of dropping down to C for any halfway-important library is why PHP's so hard to beat in real-world use, speed-wise (whatever its other shortcomings).
For the most part they don't need to know about concurrency, memory efficiency etc, because they're using a library where those issues have been abstracted away.
I think that's what makes python ideal - it's interoperability with other languages and library ecosystem means less technical people can produce good, efficient work without having to take on a whole bunch of the footguns that would come from working directly in a language like c++ or Rust.
> they're using a library where those issues have been abstracted away.
I work in Python, and while libraries like numpy have certainly abstracted away some of those issues, there's still so much performance left of the table because Python is still Python.
Julia, to use the great-great-grand-op's example, seems to also have a reasonably easy C interop (I've never written any Julia, so I'm basing this off skimming the docs, dunno, it might actually be much more of a pain than it looks like here).
https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...
Plus that you now need to write performant (and safe) C code, which (to me) defeats part of the reason to use Python in the first place.
Because if you compare the benefit to the cost of rewrite from py to C and cost of maintaining/updating C code and possible C footguns like manual memory safety, etc - then there is no benefit left
#-of-users * total-time-saved-per-user > time-spent-optimizing
Then it's worth it. You can even multiply by cost of user per time unit and cost of developer per time unit, to see how much money was saved.
Even in cases where its the same person on both sides, it can still work out. There's an xkcd comic about it, even.
And the parent was talking about numpy code which is better than stock python, who knows how far back normal python would send you.
They're comparing numpy (SIMD plus parallelism) with straightforward C code and getting a 2-5x improvement.
But there's two schools of thoughts I've heard from people regarding how to think about these bottlenecks:
1. IO/network is such a bottleneck so it doesn't matter if the rest is not as fast as possible.
2. IO/network is a bottleneck so you have to work extra hard on everything else to make up for it as much as possible.
I tend to fall in the second camp. If you can't work on the data as it's being loaded and have to wait till it's fully loaded, then you need to make sure you process it as quickly as possibly to make up for the time you spend waiting.
Not everything can be pushed into numpy, and you can still be left with lots of loops in python.
To say nothing of adding yet another long dependancy chain to the language (python 3.11 is still not supported even though work started in Aug of last year).
I do wonder if the effort put into making this slow language fast could have been put to better use, such as improving a language with python's ease of use but which was build from the beginning with performance in mind.
Could you expand on this in the context of Rust?
Please don't read this as "rust is difficult and bad", I definitely don't think it is! But its a low level language, and working with it means dealing with complexity that for some tasks just might not be relevant.
Don't forget the footguns of working with developers who do those things. Ask them to do something simple and you get something complex and expensive after months of back and forth about what is wanted. You're likely to a framework for a one off SQL query.
I hear it being said already, "You're using software developers wrong!" Well, maybe software developers shouldn't be so hard to use?
This whole take assumes bad intention on both sides. Nobody's job is easy in this situation. Leadership's job is to set everyone up for success. If things go off the rails and end up with months of back and forth leading to nobody being happy despite good intentions and honest effort, then the problem lies with leadership.
In my opinion, this is the part that Go got mostly right. Concurrency is handled by the runtime, and held behind a very thin veil. As a programmer you don't really need to know about it, but it's there when you need to poke at it directly. Exposing channels as a uniform communication mechanism has still enough footguns to be unpleasant, though.
In an ideal world, I should be able to decorate a [python] variable and behind the scenes the runtime would automatically shovel all writes to it through an implicitly created channel. Instead of me as a coder having to think about it. Reads could still go through directly because they are safe.
If I could have Python syntax and stdlib, with Go's net/http and crypto libraries included, and have concurrency handled transparently in Go-style without having to think about it, that would be pretty close to an all-wishes-come-true systems language. Oh, and "go fmt", "go perf" and "go fuzz" as first-class citizens too.
Someone else in this thread brought up the idea of immutable data structures as a default. I wouldn't mind that. Python used to have frozenset (technically it still does but I haven't seen a performance difference for a while), so extending the idea of freeze()/unfreeze() to all data types certainly has appeal.
This myth wasn't even true many years ago, it certainly isn't true today. You can build a mobile app, game, distributed systems, OS, GUI, Web frontend, "realtime" systems, etc in Python, but it is a weak choice for most of those things (and many others) let alone the second best option.
The saying means that for any given problem, there is a better choice, but second best is the language you know which has all of the tools to get the job done, so the answer is probably just a bunch of pip installs, imports, and glue code.
It’s kind of like “the best camera is the one you have with you” — it’s a play on the differing definitions of “best” to highlight the value of feasibility over technical perfection.
What worries me, though, is that the features that make Python quite good at prototyping make it rather bad at auditing for safety and security. And we live in a world in which production code is prototyping code, which means that Python code that should have remained a quick experiment – and more often than not, written by people who are not that good at Python or don't care about code quality – ends up powering safety/security-critical infrastructures. Cue in the thousands of developer-hours debugging or attempting to scale code that is hostile to the task.
I would claim that the same applies to JavaScript/Node, btw.
However, I hope that these C applications are written by people who are really good at C. I know that some of these Python applications are written by people who discovered the language as they deployed into production.
If the masterful C developer still insists on using a language that has so many footguns and a weird culture of developers pretending that they’re more capable than they are, then their C mastery could very well’ve not been worth much against someone throwing something together in Python, which will at the very least immediately bypass the vast majority of vulnerabilities found in C code. Plus, my experience with such software is that the sort of higher level vulnerabilities that you’d still see in Python code aren’t ones that the C developer has necessarily dealt with.
How could we check?
But I fear that the same folks that decried the use of excel by "the masses" are now just as horrified by the widespread usage of Python! :-)
What's an example that makes it bad? Is it a case of the wrong tool for the job?
For example I understand that garbage collection languages shouldn't be used with real time systems like flight controllers.
Python is designed to be highly dynamic and to allow more code paths to be taken at runtime, through interpreting and reacting to the live data - "late binding" in the lingo, as opposed to the "early binding" of a Rust or Haskell, where you specify as much as you can up front and have the compiler test that specification at build time. Late binding creates an explosion of potential complexity and catastrophic failures because it tends to kick the can down the road - the program fails in one place, but the bug shows up somewhere else because the interpreter is very permissive and assumes what you meant was whatever allows the program to continue running, even if it leads to a crash or bad output later.
Late binding is very useful - we need to assume some of it to have a live, interactive system instead of a punchcard batch process. And writing text and drawing pictures is "late binding" in the sense of the information being parsed by your eyes rather than a machine. But late binding also creates a large surface area where "anything can happen" and you don't know if you're staying in your specification or not.
This is very useful when you're writing unit tests or when you want to monkey-patch a behavior and don't have time for the refactoring that this would deserve.
On the other hand, this means that a module or class, no matter how well tested and documented and annotated with types, could be entirely broken because another piece of code is monkey-patching that class, possibly from another library.
Is it the case? Probably not. But how can you be sure?
Another (related) example: PyTorch. Extremely useful library, as we have all witnessed for a few years. But that model you just downloaded (dynamically?) from Hugging Face (or anywhere else) can actually run arbitrary code, possibly monkey-patching your classes (see above).
Is it the case? Probably not. But how can you be sure?
Cue in supply chain attacks.
That's what I mean by auditing for safety and security. With Python, you can get quite quickly to the result you're aiming for, or something close. But it's really, really, really hard to be sure that your code is actually safe and secure.
And while I believe that Python is an excellent tool for many tasks, I am also something of an expert in safety, with some experience in security, and I consider that Python is a risky foundation to develop any safety- or security-critical application or service.
Immutability would be the default, but mutability would be allowed, marked in some concise way so that it was easy to calculate things using imperative-style loops. Pervasive use of immutable instances would make it impossible for libraries to rely on mutating objects a la SQLAlchemy.
The language would be statically type-checked, with optional type annotations and magic support for duck typing (magic because I don't know how that would work.) The type system would prioritize helpful, legible feedback, and it would not support powerful type-level programming, to keep the ecosystem accessible to beginners.
It would still have a REPL, but not everything allowed in the REPL would be allowed when running code from a file.
There would be a strong module system that deterred libraries from relying on global state.
Support for at least one fairly accessible concurrency paradigm would be built in.
I suspect that the error system would be exception-based, so that beginners and busy people could write happy path code without being nagged to handle error values and without worrying that errors could be invisibly suppressed, but there might be another way.
If someone who's not familiar with programming runs into an error like "why can't I change the value of X" that might take them multiple hours to figure out, or they may never figure it out. Even if the error message is clear, total beginners often just don't know how to read them and use them.
They provide longer term advantages once your program becomes larger but the short term advantages are more important as a scripting language imo
I don't think mutability by default is necessary for beginners. They just need obvious ways of getting things done. There are two places beginners use mutability a lot. The first is gradual transformation of a value:
line = "The best of times, the worst "
line = line.trim()
line = line[:line.find(' ')]
word_count = 0
for line in lines():
word_count += num_words(line)
word_counts = [num_words(line) for line in lines]
# or word_counts = map(num_words, line)
word_count = sum(word_counts)
word_count @= 0
for line in lines():
word_count @= word_count + num_words(line)
It's a common claim that immutability only pays off in larger programs, but I think the mental tax of mutability starts pretty immediately for beginners. We're just used to it. Consider this example:
dog_name = Name(first='Rusty', last='Licks')
dog1.name = favorite_name
dog_name.last = 'Barksalot'
dog2.name = favorite_name
print(dog1.name) # It's not Rusty Licks!
Julia is both dynamic and fast. It doesn’t solve all issues but uniquely solves the problem of needing 2 languages if you want flexibility and performance.
Coupled with the dynamic typing and mutability by default, it guarantees Python programs won't scale, relegating the language to the role of a scratchpad for rough drafts and one off scripts, a toy beginner's language.
None of those applications are toys. I have no idea where your misperception is coming from.
All sorts of problems would simply have not existed if the proper language was used from the beginning, as opposed to the one where anyone can hack most easily.
It’s already what you get with python and mypy. Using typing.Protocol or Unions.
Regular sofware development is a lot like that as well. But you can kind of get around that by having Python as the "prototyping language", and anything that's proven to be useful gets converted to a language that's more useful for production.
Do we agree that this somewhat orthogonal to what I'm writing, though?
Most of the provided reasons behind Python's popularity are true also for other languages - portable, open source, productive, big community. This can be also said about PHP, Ruby, or Perl back in 2000s. Why isn't Perl as popular as Python?
I don't think it's all about readability or productivity, but about tools that were built over the last 30 years that have been used in academia and now with the boom in ML/AI/Data Science, they made Python an obvious choice to use for the new generation of tools and applications.
Imagine that the boom in ML/AI didn't happen - would Python be #1 language right now?
Python was friendlier for beginners than Ruby the first time I took a real stab at learning to code during a CNY holiday in 2008, but it wasn’t about the language itself. Ruby was harder then because many of the popular libraries and many of the tutorials were written by people who considered Windows support as an afterthought. It’s hard to express how frustrating it was to have my vacation days ticking down, hitting issues in one tutorial after another and having people suggest I install linux on a VM (a process where I hit still more snags).
People learning Python and PHP didn’t hit that hurdle. I ended up learning Flash on my Asus laptop a couple of years later and getting my start that way and not coming back to Ruby until six years later when I was a much more experienced dev.
I don't think there is a single reason, but it sure didn't help that the community self-destructed by trying to make an entirely new language after version 5 and still call it Perl. It took a lot of years to resolve that nonsense, and in the meantime many people moved on.
It also does not help that Perl is a creative language, useful but very much open to many different interpretations. Hiring a perl guy and expecting them to read someone else's code is a crapshoot. The upside to Python's strong cultural opinions on coding style makes it easier for one developer to pick up someone else's code.
> Imagine that the boom in ML/AI didn't happen - would Python be #1 language right now?
Probably not. But it wouldn't be perl, either. Javascript most likely. But the core usage of python for scripting was never predicated on ML popularity, so it would still be a pretty commonly used language. and javascript has many annoying warts too, so I think plenty of people would still choose to write django apps instead of node, whether ML existed or not.
One of the few programming language jokes I've liked enough to repeat is "Python in Perl for people who can't bring themselves to write Perl code"
Perl was significantly more popular at one point, but it slowly lost traction while Python gradually gained traction over the years.
Better ecosystem for numeric computing is definitely a big reason for the success of Python, by the question is why Python gained a foothold in that niche in the first place. It think it is because Python is just a lot more accessible to people with different backgrounds. Perl really grew out of shell scripting as a supercharged alternative to Bash and Awk, but retaining many of the quirks for familiarity. Python on the other hand grew out of research in teaching programming to beginners.
This is a strange article. It's got the talking point about Python that we were hearing about 10 years ago - "tired of those pesky curly brackets in Java, try this new language you might not have heard of: Python!". Who reading the GitHub blog has not heard of Python?
Also, that snippet used in the "What is Python commonly used for" section is strange:
import antigravity
def main():
antigravity.fly()
if __name__ == '__main__':
main()
It then ends with what seems like it might have been the point of the article, an advert for Copilot. It seems the way to get started writing Python is to write a short comment and then spam <TAB> and let the AI auto-complete your project.
(Also, and perhaps less importantly, looking at the author's GitHub profile I can't see a single instance of Python there. Though I'm not doing a deep-dive as that feel overly picky and there's plenty of private contributions that could well be Python.)
That was a talking point closer to 20 years ago, at this point.
1980-2000 = "30 years ago"
2000-2010 = "10 years ago"
2010-2023 = "RecentlySyntaxError: Missing parentheses in call to 'print'. Did you mean print("Hello, world!")?
https://stackoverflow.com/questions/6182964/why-is-parenthes...
https://github.com/python/cpython/blob/main/Lib/antigravity....
In the end it is just an ad for Github products and not worthy of being on HN frontpage.
The GitHub blog became a strange place recently(-ish). It went from a factual blog describing fancy new GitHub features and interesting technical stuff (as it was a decade ago) to mostly a place full of incoherent marketing fluff like this post (with some real technical content interspersed).
import antigravity
pydoc3 antigravity
IMO, that example is there to show that there's less required boilerplate required in python compared to java when doing the same thing. And, in particular, none of that boilerplate really matters to what you want to do - print hello world - emphasizing the point of python being simple.
That's not a sales pitch for python, it's a sales pitch for the concept of a scripting language; it's like saying "you should really buy a Ford, it comes with four wheels".
I wouldn't say it's friendlier than the alternatives, Perl and Shell scripts sure, but not when compared to Javascript, Ruby or Lua.
Now, if you're talking about libraries, support, etc. then sure, Python wins hands down, but that doesn't make it a better language in itself. I'd say Ruby and Lua are a little bit better as languages.
But then again, I don't care much for the language in itself, so Python is enough for most of my use cases.
In my job I've seen some beginners starting with R, and quickly hating it because they don't feel they can do much on their own, but copy-pasting and then modifying from the examples and the tutorials. And it the changes go too far, everything collapses with cryptic errors. When you show them Python as an alternative, pointing that they shouldn't use it over R for statistics and graphics, they like that they can build ideas from the scratch. That beginner is hooked for life.
The story of Ruby is altogether different: they made the fatal mistake of not defining a C foreign function interface in the standard, otherwise I imagine we'd be seeing numerical computation and ML libraries with a Ruby interface today. Still, Ruby lives on in Metasploit, and in Sorbet and Crystal.
This to me is extremely plausible, and sad.
- Lua's standard library is so weak that it makes most other batteries-not-included languages look like they have large, robust, and helpful standard libraries.
- It's got a bit of the quirkiness and gotcha-ability of JavaScript but without its being a language that's impossible to avoid due to capture of a mega-popular platform, which is what propelled JavaScript to ubiquity despite its being kinda shit and unpleasant to work with.
- Tooling's not as good as many other languages.
(FWIW sometimes I write Lua regardless, because it's the right tool for the job)
In fact I'd argue that python enjoying the success it has, despite probably the worst handling of a version bump in any language (2->3), is a testament to its popularity.
Ruby would have a much bigger market share these days if library path dependence was that powerful.
Chiefly, because big money corp invested massively elsewhere.
Also by now it’s easier to find developers who are cheaper and already (only can) use javascript/python?
I think that actual technical merits are dwarfed compared to other forces at stake here.
https://github.com/elixir-nx/nx
I don't see how it could ever overtake Python, but it could establish itself as a viable niche alternative.
This week our Director ordered a total rewrite of two years of work in Python. His rationale: it's what everyone else uses in this space. No reason specific to our use case, just simply to follow the herd. I realise that a large community translates into easy hiring and rich ecosystems, but I despise the mentality as it promotes a monoculture.
There was a time when all you had to learn was Java or C++ or (language or tool here). Somehow, this time when we standardize, it will be different.
= be usable, and
= provide a set of data structures that an educated programmer *expects to find when scripting*.
Can you say the same about your solution?
WTF? Unless your system is originally written in a proprietary language that literally no one outside your company knows, I'll say it's a good sign that you need to change team (or change job). Don't work under a director like that.
Better luck with the next one!
ES modules, fat arrow expressions, and all the other nice new syntax and library features have made the language so more pleasant to use. In many ways the ergonomics of TypeScript in particular are far superior to Python now, and I find that really surprising.
I haven't yet found a replacement for Django (particularly the ORM/Admin/Forms combination) it is just so incredibly productive to use. So, I don't think I will be moving off it, but it's certainly not the growth language for me anymore.
This, plus Typescript's superior type system makes it very tempting for application development.
However I'll probably wait until there's a really good Jupyter Notebook equivalent...
https://github.com/n-riesco/ijavascript
I used this for interactive debugging AWS API calls using the AWS Javascript SDK. It worked great!
Closest thing to this is Docker nowadays. It's so annoying. npm just works.
I still love Python. Just feel like I'm cheating on it with this new younger model...
Take a look here https://adonisjs.com
- Particular support for web frontends or backends, both very broad categories.
- JS concurrency is easier to deal with. Focused entirely on promises with the nice async/await syntax on top, unlike Python which slapped on too many different ways to do this.
- By far easier package management and imports. Python's is so annoying that any project you download is gonna have you spin up a Docker container for it.
- ES6 added a lot of array/etc managing that JS was lacking before.
- Freeform objects with {key: value} syntax are convenient, despite maybe seeming weird at first. Python OOP somehow got really complicated over the years.
- Inline functions (I use fat-arrow but regular way is also fine). I never got why Python, despite being common in function-oriented programming, didn't let you do an inline def.
- Curly braces. Indentation shouldn't affect logic flow.
- No Python3 vs 2 drama.
Specific use cases have been Express backends with node-postgres and lots of SQL, and React (or RN) frontends.
Someone else made this remark once, when the indroduction of type annotations was discussed as the python community discovering the benefits of static typing.
Another major factor is how it serves as a glue language in the scientific community. Python provides a relatively simple programming language that powers complex, powerful libraries like NumPy, SciPy, PyTorch, SymPy, etc. It's like a scripting language for engineering and scientific computing tools, sort of like what Javascript does for browsers, Node, and Deno, etc.
I don't do much Python programming these days but I owe it a huge debt! May it continue to flourish and grow.
Not to rag on Python, but I found the indent-sensitivity of Python can make it challenging to do the sort of basic things that C-family languages make fairly simple. Lambdas in Python suck in contrast to how lambdas and anonymous functions work in other languages.
But yeah, Python has proven itself to be a great language for a wide variety of applications. It makes a lot of sense for the scientific community probably because things like heavy object orientation matter a lot less in those cases. The community doesn't discourage people from just writing functions, unlike other communities that are hellbent in making everything a method of some class structure and being explicit about that.
And it’s not very portable the second dependencies with native code (which is common since pure Python is too inefficient for many tasks) are used.
I think Python is popular for two reasons:
- it’s believed to be beginner friendly compared to other languages. I’m not really sure why - maybe the whitespace?
- it has an enormous set of libraries available to use
In other words, Python has momentum. The language itself isn’t great, but that is a secondary concern to most, if they are even aware of this at all!
I am leaning more towards it looking like pseudocode
> - it has an enormous set of libraries available to use
I believe that to be the actual reason. I've long held that python is a kinda bad language, but with FANTASTIC libraries
ftfy - it's the only way a tragically slow language like Python can keep up.
Edit: didn't forget FORTRAN
That is simply not true. Even libraries like numpy, scipy and scikit-learn are majority python code.
https://stackoverflow.com/questions/1825857/how-much-of-nump...
For the vast majority of use cases, performance just isn't a priority. Doubly so for Python, that shines for simple automation, command line applications, and perhaps some serveless computing.
Being easy to write, having a good ecosystem of libraries, and being widely known is typically good enough. I wouldn't use Python to write a robust backend server side application, mostly because the language doesn't lend itself well for it.
If it was too slow, we'd be doing all of this in Java, the C# or maybe doing it in C/Fortran. But because of some early design decisions (Guido being on the matrix-sig helped), the history behind Numeric/Numarray and finally NumPy and SciPy being based on those efforts allowed it to thrive.
Those were your words, not mine. I need not make any assumptions.
I just replied listing use cases where Python shine due to its strengths, performance being mostly irrelevant. I didn't even mention data science.
And although it's beyond the point, if I was to use Python, why should I care in which language a library was written? If the language allows libraries written in other languages, this is actually a nice feature.
That’s actually my primary use case for python, playing with C/C++ libraries in a repl because they don’t natively have one.
Sure, it takes work to wrap a library but that’s something I enjoy doing.
Just wait until they learn the python interpreter is written in C… <grabs popcorn>
the best teaching language (imo) is one with few features or surprises. I think Scheme might come out top here.
Depends what you are trying to teach. If you're trying to teach computer science and programming fundamentals, then sure. If you're trying to teach people how to get 'real work' done quickly and efficiently then Scheme will only get in the way and slow people down.
For example when I've taught programming, one of the tasks I taught fairly beginner programmers was to grab some satellite images between certain dates, try to detect if there is a forest fire, measure the spread of the fire and plot the spread on a map.
With python (and its excellent libraries) this is quite quick and easy, and most people are up and running and hacking around with their programs in pretty short order. They find it really cool and inspiring and makes them quickly realise that programming could be something useful in their day to day job. Trying to start with chapter 1 of SICP and Scheme and working up from there to solving the above problem would probably lead to most of these people giving up on programming very quickly. That being said the few people that made it all the way through that would no doubt be much much better programmers because of it.
It looks like English. That's all pseudocode is, and they know that well enough.
If we want to measure developer productivity we should try to compare actual real world usage that goes beyond "hello world" using Notepad. The author did provide more examples in the article, but I think we should just retire the basic minimal hello world example for these types of discussions.
If their argument is that Python is simpler for quick scripts and programs, I agree. The same applies to ML / AI where Python has lots of great tools. Once you start looking at other areas (Django and FastAPI), there are many alternatives based on other languages where you can be just as productive.
Most of the time consuming work is spent doing business logic where the differences between languages doesn't matter as much. They all have advantages and disadvantages. Personally I prefer static typing for larger projects and teams.
If their argument is that Python is better in general, they need to provide better arguments.
Good bait. I'll take it.
- dynamic, weak (really "duck") typing, meaning users don't have to worry about conversion between things. Want to print() a dictionary of whatever? Sure!
- no semicolons to terminate a statement. End of line, that's it
- rich standard library, so you can actually get going on things without having to go on a quest to find the right library for your thing. JSON? It's right there. argument parsing? argparse. Http? http.client works...
- also yeah the "pseudocode" thing. Python is light on extraneous syntax.
Now eliti^H^H^H advanced programmers will frown on many of these same things that make it beginner-friendly...
Dynamic typing means that complex designs have to be careful with their APIs, lest you can get tangled up in deep type errors if you're not careful (I particularly hate the "mix-in" pattern). No semicolons mean, uh, long statements need to escaped? The standard library is "where lib go to die" because they can't evolve as much. "Significant whitespace!? What is this, COBOL!?!11"
Ability to do structural printing of arbitrary structures is not restricted to dynamically typed languages.
However, I actually think a good beginner language could be dynamic or static.
import request
data = request.get(https://some.api/get_some_json)
some_value_I_want = data['dig']['into']['nested_structure']
Also, say I want to make a nice plot of some tabular data (.tsv), I do:
import pandas as pd
import seaborn as sns
data = pd.read_csv('some_file.tsv', sep='\t')
pd.melt(data, value_vars=['s', 'max_vms'], id_vars=['sample', 'process', 'N'])
g = sns.FacetGrid(data=data, col='process', row='variable', hue='sample', sharey=False, margin_titles=True)
g.map(sns.barplot, 'sample', 'value', order=data['sample'].unique())
(Btw, I'm putting 4 spaces in front of the code, why is it not rendered as code?)
When I just started I used Python to read, sort and transform images and output them to a PowerPoint file. One can use CSS like synthax to format the slides. Boom PPT with 120 slides with images and data from some Excel file that convinced a lot of people with a lot of data that fluorescent images next to H&E stains + metadata can be nice. Is it the best way to present such data? Meh. But man was it cool and easy and time saving.
Maybe it also doesn't help that out there, in the C++, Rust, Javascript, Go world I'm going to run into people with usernames like "ihatepython".
quote_from_user_ihatepython == """ ihatepython 1 day ago | parent | context | prev | next [–] | on: Moscow Metro launches longest metro circle line
You realize that the Nazis still exist and are Ukrainian, right?It is however what a lot of people who 'program' at work do every day. Python actually lets working professionals solve real problems they actually have at work quicker and easier than any other programming language.
But, with a user name like that, I know I’m just wasting my time responding to you.
But what if your web request fails? Do you deal with it in time or use some_value_I_want which might contain junk?
Python does have a great data processing ecosystem. But that isn't really a property of the language.
Some languages have a good integrated tool chain (Rust, Go) but are not as approachable and forgiving.
Some are approachable but lack the friction-free story for locating and installing 3rd party dependencies.
Someone else identified Ruby's fatal flaw as not having good C tooling, and I think that's probably accurate.
Python definitely found footing based on its pseudocode readability. It’s gotten a bit worse recently with too many colons and features but already made it.
It shows an obvious reason why a lot of beginners choose python, even when learn Java basics in university first.
1. Near zero boilerplate. Python's boilerplate is usually no more than setting up a class and then calling a method. Often you can skip the class and just call it directly. This is probably the biggest strength; to give you an example, my standard test for a languages approachability is "how much work do I need to put in to get a very simple JSON file from a web URL" (nothing fancy like POST, just an HTTP GET). With python, a call to urllib.request.urlretrieve and then a call json.loads are all you need. In Java, you need to manually implement a bunch of boilerplate code that looks horrible, need to think about the size of your response in memory and often need to pass in configuration options that should be a case of "works by default" but aren't because the standard is ages old so it needs to be manually toggled on. Part of that is that the Java stdlib consists largely of reference implementations rather than actual implementations, which means that anyone who wants to implement something in Java will usually end up falling back to the stdlib interfaces, which in turn suffer from being stdlib interfaces, so a lot of "should really be on by default" expectations aren't on by default since the stdlib is where code goes to die, so you instead start piling on features.
2. Interop with C. I hold the opinion that C is a great language that doesn't work very well once you get to any form of scaling. Python allowing developers to take the slowest parts of their code and writing it in C to speed it up is one of the easiest speed gains to make and it avoids the biggest bumps that come with using C as your primary language in terms of project structuring.
3. Library support is as you say, good. If you need it, there's probably a package for it. Pypi is dependency-wise kind of a disaster but if you know how to set up requirements.txt, it works really well. Most libraries ship with "sane" defaults too.
All of these combine to a language that's easy to prototype, easy to expand
It's important to keep in mind that pythons biggest successes aren't in the speed-focused, low-memory environments where every speedgain is necessary. Its success lies in conventional desktops and servers, which have much more processing power and often have more leniency in being a bit slower.
With python you can write something in 3 hours what would take a day in another language, at the cost that instead of being lightning fast and done in 10 seconds, you need to wait 30 seconds. That's an issue for some environments but in 99% of the cases that's not a problem.
That isn't to say the language is perfect (no language is), but speed of development at the cost of slightly slower execution time is the main reason why python got popular.
In C# you just have to do this: var things = await httpClient.GetFromJsonAsync<List<Thing>>("url");
Most languages nowadays tend to implement some variation of this specific convenience because it's just one of the most frequently needed things; setting up a separate parser and then calling it might be the "cleaner" option, but it's also more boilerplate and the industry has largely moved to try and avoid that.
0: https://learn.microsoft.com/en-us/dotnet/csharp/programming-...
Well yes it it does now, since C# 10 or, but in Python is have been a one liner since the beginning 30 years ago, and over the decades it has build a following. And even with recent efforts to cut down on boilerplate, C# still has more cryptic syntax than Python.
I'd argue C# is more maintainable in the long run due to the static typing, but there is no way it is as accessible to beginners.
Experienced programmers tend to forget how the experience was as a beginner programmer. But Python grew out of actual usability research into teaching programming to beginners. Logo is another language with the same background, but it never escaped that niche. The success of Python is because it attractive to beginners but remains a powerful tool as the programmer becomes more experienced.
Whitespace is definitely a factor - or rather the lack of redundant braces. Beginner programmer seem to really struggle with the lack of correspondence between the visual structure and the logical structure in most languages. Even experienced programmers get tripped up by erroneous indents. Python just solves this once and for all.
The lack of type annotations is also a factor. For beginners this is just an additional layer of complexity.
Scheme is great for teaching computer science students, but if you are just a regular Joe researcher wanting to get the job done, you want to write "2 + 2" like everybody else in the world, not "(+ 2 2)"
Most languages are designed to attract experienced programmers, e.g. by having C-like syntax which they may already be familiar with. But it has a cost for beginners.
You're not forced to dip into the OO side of things though and I've heard people suggest to me that Python is a language where no rules apply and no structure exists. They feel that they can dive in without the care they would have to take in Java or C++. I think they are very wrong but you can certainly write terrible code if you want and perhaps this "all-things-to-all-people" aspect of it is part of the success.
Understandable code is maintainable code. Abstraction makes comprehension possible — imagine if every call to print were an inline block of assembly instead! — and Python’s modules provide a very quick and easy way to break up large code into small modules.
The point of communication is to express something in a way that the listener understands it. Simply expressing something that is comfortable or normal to the speaker is simply not useful for either party of the goal is communication.
> They didn't pay too much attention to white spaces and tabs, probably because they are "invisible"
can get so much worse than people imagine, when the language doesn't enforce it. It was that experience that made me lean towards python being a good introductory language, to help people get used to correctly indenting code in general.
I had a BMP file which I wanted to transpose and turn into raw RGB 5-8-5 values. Using the Python Image Library made things absurdly easy.
That's Python's advantage in a nutshell. Things are just so absurdly easy.
If there was an equally good C++ image library available then it would have been equally simple in C++.
Of course the thing is that, in general, there may well NOT have been such a C++ library, and this is what makes languages such as Java and Python popular - because of the breadth of libraries available.
For C++ image manipulation I've found libvips to be decent, but the point still stands that for a given problem there may not be decent libraries available.
Most of these domains are sort of document-oriented - for the math stuff, the hard part is just defining the model, and there is only one "thing" to operate on. For testing, the units of work are test cases and steps, which _could_ be database entries, but work better in practice as a document that a non-technical QA team member could edit by hand. Results are fed to a SaaS that keeps all the historical test result data.
Not that I couldn't pick up either one of these tools and use them (I'm a mechanical engineer by training and got into Python because I didn't like using Matlab/Octave), I've just never needed them professionally. But that doesn't get me past the resume filters :(
Also, Python makes it easy to work in different programming paradigms, it's as easy to write functional-type code as it is to write object-oriented code, both approaches are supported. This makes it a great prototyping system if you want to rewrite the code later in some other language like C++ to get some performance improvements.
Launching processes and interacting with them through stdin and stdout is a bare minimum, and really a flag against the few languages that make it hard, instead of a relevant feature.
> Python makes it easy to work in different programming paradigms
As long as you stay in imperative, non-pure, structured with optional OOP and first classes functions. Again, that's not much in multi-paradigm. This is the set of things that work well with the imperative model.
I think the thing I hate the most about it is that white space is significant. It's like we travelled back in time to the early days, and picked up one the bad things about them and brought it back. I find that makes it more difficult for me to read, and more difficult for me to write (from a mechanical type-in-the-code point of view).
I have other niggling issues with it, but if the white space thing weren't an issue, I doubt the other issues would bother me enough to complain about them.
If you’re having trouble with them, use a programmer's editor with indentation guides like notepad++ or geany. These are considered basic features these days.
Other simple things which reduce Python annoyances are tools like pyflakes and the blue formatter.
> If you’re having trouble with them, use a programmers editor with indentation guides like notepad++ or geany.
Yes, of course. Python makes that pretty much essential, which is one of my complaints about Python. A language that requires a special editor is a language that is deficient, IMO.
There are a ton of other, more minor, aspects of Python that makes it unpleasant for me. It's not just the whitespace thing. That's just the one that grinds my gears the most, because it's the one that slows me down the most.
But note what I haven't said here -- I haven't said that Python sucks. I only said that I hate using it, despite being pretty fluent in it and having used it for years.
A lot of folks can’t survive today without a giant ide awhile you advocate not to use something downright tiny in comparison. Do you hate working with jpeg or blender files because they require software? Rather move bits with a magnetic needle and steady hand?
Admit I was put off by it for an hour or so as well. One spring day in 2001? That afternoon I realized it as a masterstroke and didn’t write another line of c, perl, or java by choice for almost two decades.
Two ways to delimit blocks is redundant. Either one indents already or the project is a disaster.
I just wish that I did.
> A lot of folks can’t survive today without a giant ide while you advocate not to use something tiny in comparison
I'm not advocating anything. I'm stating my personal preferences. But I also make sure that I don't rely on a giant IDE for any language. I use a couple of different ones at work, but I don't use one at home (even though my hobby projects are no less complex), because I've found that using IDEs encourages me to engage in poor programming practices. I am not saying that nobody should use IDEs or that they make people worse programmers. I'm speaking for myself. I want to deeply understand the languages I use.
> Do you hate working with jpeg or blender files because they require software? Rather move bits with a magnetic needle and steady hand?
Of course not. Those are not human-readable data collections are need a tool to make them understandable by a human. A programming language is supposed to be directly understandable by a human, though, and if you need a tool to make it so, that strikes me as a failure of the design of the language.
> Two ways to delimit blocks is redundant. Either one indents already or the project is a disaster.
Eh, each to his own. Yes, there is a redundancy there, but I think it's a redundancy that brings value and reduces error.
Not sure how well-known but many 3D scene formats are text, especially early ones. Similar to svg conceptually, which many are familiar with. At a certain complexity, writing it by hand is no longer practical and software support approaches necessity.
It boils down to "the right tool for the job", of course, and like every developer, I choose my tools with an eye toward the goals I want to accomplish. That my goals and yours aren't 100% aligned is to be expected. We're different people. And that also means that at times, a tool that is appropriate for your use case may not be appropriate for mine. And vice versa.
It isn't redundant though because without delimiting symbols for a code block you lose the ability to have your code autoformatted in certain situations. Here's a trivial example to illustrate the point:
def example():
x = 5
print("Hello world")
function example() {
let x = 5
console.log("Hello world")
}
I see how a formatter could help you out in this specific situation. However you are trading typing of redundant characters every few seconds and readability per minute, to avoid an issue that happens once or twice a day, per week on a mature project.
In other words we generally don’t significantly reorganize code nearly as much as reading, tweaking, adding features etc.
Readability is definitely a strength of Python, not a weakness. A lot of this is because of reduction of required notation.
The code runs so it's most definitely not a syntax error. May be against PEP styling rules, but it is valid Python code.
>which must be fixed, and takes a second.
And that was my earlier point. The responsibility to get the code in a state where it is formatted AND runnable falls entirely on you, as this work cannot be entirely delegated to software when the syntax uses significant indentation. And the fixing of the code would require you to reanalyze the code's semantics before you would even know what the appropriate fix is. Of course it would only take a second for a trivial example like the one I used, but real Python codebases aren't going to be trivial.
>typing of redundant characters
It actually doesn't require any additional typing as compared to Python. In modern editors the closing brace is automatically added when you type the opening brace. So you simply type the opening brace and hit enter, just as you would type the colon and hit enter in Python. Even the space between the closing parenthesis and opening brace in the function header is added automatically by formatters.
>readability per minute
Seems pretty subjective but I don't think there's a significant difference in readability between Python and braced languages, or even between Python and languages that use block delimiters other than braces, like Ruby. A lot of readability comes down to personal familiarity with a language.
>we generally don’t significantly reorganize code nearly as much as reading, tweaking, adding features etc.
Totally agree, and I think this is one of the big problems of software development. People generally don't want to make big reorganizational changes to code and instead prefer to change the code only through additions. As a result, legacy projects tend to accrete layers of cruft over time whether it is necessary or not. I'm not a Jonathan Blow fanboy by any means, but I recently saw this clip which I think makes a good point. https://www.youtube.com/watch?v=ubWB_ResHwM
$ python3
Python 3.10.6 (...snip...)
>>> def example():
... x = 5
... print("Hello world")
File "<stdin>", line 3
print("Hello world")
^
IndentationError: unindent does not match any outer indentation level
It's idea is sort of neither here nor there however, regarding whitespace blocks. That we "rent" more often than own is just a reality of the system we find ourselves in. (For example cheap products sell more than expensive ones and that is expected and ok.)
I don't want to optimize my projects around large refactors since I only do it a few times per project, but will read it often.
$ python3 foo.py
File "~/Desktop/foo.py", line 4
print("Hello world")
^
IndentationError: unindent does not match any outer indentation level
def foo():
bar = 3
print ('4')
>>> def foo():
... bar = 3
... print ('4')
File "<stdin>", line 3
print ('4')
^
SyntaxError: invalid syntax
def foo():
bar = 3
print ('4')
>>> def foo():
... bar = 3
...
>>> print ('4')
4I find Python to be on the lower end of average in terms of readability.
For the most part these update without much hassle, but now you've "broken" your researcher's code. It takes them a while to fix this, and then the security guys find out that Python packages also need updated...
Having JavaScript in the front end and backed both can reduce resource requirements as well.
It'd be interesting to see an attempt to assess the growth in use of Python outside of AI/ML.
The languages are practically equal in terms of features. They're both typeless with layered-on crutches available to make the runaway dynamism less painful. They both have weird footguns and ugly syntax and annoying design flaws, but these are different for each. So if you're forced to use both languages, it's an endless pain in the ass to remember which stupid runtime error can happen in JS but not Python and vice versa.
So JS can replace Python... But Python can't replace JS simply because it's unavoidable on the front-end. Since you must have JS code anyway, why use a language that's functionally very similar but different enough in syntax and API that you'll be writing bugs all the time? Just stick with JS and start drinking like me.
But I don't actually want to do that. The Python design philosophy isn't really compatible with loading embedded, sandboxed user programs over the network. A hypothetical Python-Netscape would have crashed and burned with security holes like ActiveX, IMHO.
Its dependency management story is not great, but in general better than the horrible mess on the Python side.
On macOS there's a system runtime called JavaScriptCore.framework. It's actually a very good and useful engine. If you're writing native code, you could link to that. But of course it doesn't provide any of the Node.js API, just the JS standard library which is tiny. So the most typical use case would be to embed a JS API into your application.
1. Some probably usable version of Python is already on your computer. Not true for JS.
2. Forced use of async in JS library code, even though Node.js doesn't require it.
It's really hard to trace a Python bug based on codebase, due to its stricly bugly indentation sensitive for space/tabs.
I rather work on a JS codebase than a python codebase.
But i consider Python coders genius, because they can go deep into rabbit hole really well.
That may only prove that you are not proficient in either.
This article is garbage. It reads like SEO trash and I'm not sure why Github feels the need to publish it. It's just a list of common Python talking points. Points that the author displays a poor understanding of.
The article doesn't really attempt to explain why python keeps growing but the facts listed vaguely suggest that it "keeps growing" (not sure when it was supposed to die) because it is Good and Popular. Which makes sense, I guess.
If the title was '10 Cool Facts About Python (a Programming Language!)' I would probably find it a little bit funny.
import sys
def main(
args: list[str],
) -> int:
sys.stdout.write(“hi\n”)
return 0
sys.exit(main())
It’s just that you don’t have to write testable, type-safe code if you don’t want to.
Some of it is the language too, but modern Java can definitely be reasonably terse. Record classes has done a lot for the language, so has lambdas and streams.
Modules/Project Jigsaw is arguably an even better step in the right direction. It arguably allows a much saner granularity of visibility and access.
Example:
import antigravity
def main():
antigravity.fly()
if __name__ == '__main__':
main()
The only thing that isn’t perfectly straightforward is “what is __name__”, but once you know how __name__ is defined…
OTOH, that example is much more complicated than needed. As a simple executable script, that would do the same thing if antigravity. Fly() actually did anything, all that is needed is:
import antigravity
antigravity.fly()
import antigravityIts…not, actually. A more typical, and testable, example of python would be:
import sys
def main(args: list[str]) -> None:
sys.stdout.write("hi\n")
if __name__ == "__main__":
main(sys.argv)
* Isn’t testable because it runs the sys.exit() call unconditionally on import, so there is no way to import the module and call main() from test code.
* Fails with an error, because main has a required argument that it isn’t supplied in the unconditional main() call inside the sys.exit() call.
* Isn’t following a typical python idiom by returning a value from a function and have it be unconditionally 0; if the only options are a fixed return and a exception somewhere, the convention is to return None (which doesn’t require an explicit return.)
* Since a normal exit is the default response of ending the main program, sys.exit() is unnecessary here; “Exit with 0 if main() returns successfully, and with an abnormal exit on exceptions in main” is achieved by just calling main() and having nothing after it in the main program.
Easy to learn yet powerful. Easy to read yet very concise. Anytime you need something complicated, it's a pip and an import away.
Couple of downsides though: - slow - does not handle multiprocessing nicely.
If the python team ever manages to fixes these two (not easy or it would have happened a while ago) to bring it to - say - the level of Go, Python would be the absolute killer language.
But even as someone who knows C++, most of the stuff I do doesn't require it - and python is often easier and terser to express the concepts.
It'd be a lot easier (and more reliable) just to use a traditional compiler, rather than an LLM imitating a compiler.
The problem is maintaining the full scope of Python semantics there is usually no advantage to this. Compiling parts of a python codebase, with restricted semantics, can give you significant gains in some use cases, and there are tools (in Python!) that do that already.
https://mesonbuild.com/index.html
Getting python bootstrapped is less easy than getting make bootstrapped, plus python isn't a GNU tool so I don't see them adding a dependency on it to all their packages.
GNU Make is really far more sophisticated than people give it credit for and it's not that easy to replace it.
> GNU Make is really far more sophisticated than people give it credit for and it's not that easy to replace it.
Since its turing complete, it should be able to do everything make can. For example, to compile a C project involves something like:
1. grab all the C files inside `src/`. 2. join the filenames with whitespace delimiter 3. execute `gcc <join result>`
It can also be used to figure out the OS, where to find dependencies and what order compilation should happen or whatever.
But why doesn't this happen?
Make doesn't do any of the things you mention - it just works out what targets need to be rebuilt and the code to do it is in the rules that you have to write in a shell language like bash.
I admit you can use builtin implicit rules for C and get away with that up to a point but only a fairly low point.
python is far harder to use than bash when it comes to running processes and doing things with them and doing the file-system manipulation that is usually wanted when building.
There are modules that make it easier but so far as I've seen it's a crap choice plus python takes a long time to start up so there is a cost to running a "clean" environment for every build step but OTOH if you use the same interpreter for the whole build you can introduce all sorts of ordering problems when a build runs on a different machine and doesn't work because it's executing build tasks in a different order.
If you really want to build by "writing a script" (groan - because that's the age old horrible solution) why bother with python?
Can easily be solved by having libraries that does. Which is far better solution that hacking/chaining 5 different obscure unix tools to do string manipulation.
>you can introduce all sorts of ordering problems when a build runs on a different machine and doesn't work because it's executing build tasks in a different order.
Surely it is better than make? Even if make is "better" because it "just works". This is basically Hyrum's Rule waiting to be unleashed.
>If you really want to build by "writing a script" (groan - because that's the age old horrible solution) why bother with python?
Because I personally cannot decipher make and I think this is the experience of many developers as well. And how many make derivations are there? Its basically an arcane spell that needs to be chanted with every project (configure -> make -> make install).
It's not arcane - you just don't want to learn it. You can write your own system and rediscover all the issues and resolve them if you like.
If you started from understanding make, though, you could do a better job than it has done because you don't have to be compatible.
> Using Python for and artificial intelligence
for what and AI?
Try a lisp instead.
That pretty much says it all.
I branched out to C++ and other languages to stay above the curve. Python is becoming like english, required to know, and not really a deep skillset employers are looking for now.