Could? Yes. Easily? No.
People write their business logic in Python because they don't want to code in those lower-level languages unless they absolutely have to. The article neatly shows the kind of additional coding overhead you'd have to deal with - and you're not getting anything back in return.
Python is successful because it's a high-level language which has the right tooling to create easy-to-use wrappers around low-level high-performance libraries. You get all the benefits of a rich high-level language for the cold path, and you only pay a small penalty over using a low-level language for the hot path.
But if that's an abuse of the tools (which I agree with) how does that make it the fault of the language rather than the user or package author? Isn't the language with the "rich library ecosystem" the natural place to glue everything together (including performant extensions in other languages) rather than the other way around -- and so in your example, wouldn't the solution just be to address the abuse in pytorch rather than throw away the entire universe within which it's already functionally working?
If it avoids excessive copying & supports parallel computation, surely it's fine?
If your model is small enough where the overhead of python would start dominating the execution time, I mean... does performance even matter that much, then? And if it's large enough, surely the things I mentioned outweigh the costs?
I thought the point of numeric processing frameworks&languages in general is that if you can express things as common math equations, then geniuses will go in and implement the hyper-optimal solutions for you because the'yre extremely common. If anything, it should resemble scripting even more, because you want to match the structured way as much as possible, so the 'compiler' (or in this case backend C libraries) can do the lifting for you.
I've implemented KV caching in C++ and seen it implemented in Python, I see your point.
No large scale training & inference either, that's cool, if the model can't even fit onto a single GPU. I can see how memory communication can become a significant issue, since you'd have to manage that through python if you're managing python kernels. (Though you technically could just throw all the responsibility down to the lower levels yet again... not a good idea & polluting responsibilities though)
A single python interpreter stack frame into a 10^4 * 10^4 GEMM C BLAS kernel is not a bottleneck, but calling 10^8 python interpreter stack frames for a pointwise addition broadcast op would be a bottleneck.
Does pytorch overload common broadcast operations though? I was under the impression that it did as well. I guess this is what `torch.compile` attempts to solve?
It's a mistake to assume C is always faster. If you don’t have a deep understanding of memory layout, compiler flags, vectorization, cache behavior, etc. your hand-written C code can easily be slower than high-level Python using well-optimized libraries. See [1] for a good example of that.
Sure, you could call those same libs from C, but then you're reinventing Python's ecosystem with more effort and more chances to shoot yourself in the foot. Python gives you access to powerful, low-level tools while letting you focus on higher-level problems—in a language that’s vastly easier to learn and use.
That tradeoff isn't just convenience—it's what makes modern AI R&D productive at scale.
[1] https://stackoverflow.com/questions/41365723/why-is-my-pytho...
Amateurs and even some folks who have worked with these things for a while won't beat off the shelf python calls to highly optimized and well constructed libraries.
Mind you I've programmed in all the mentioned languages. ;)
Were I to write a scripting language, trivial export to .so files would be a primary design goal.
It basically forces your language to be very similar to C.
Ruby, Python, and Perl all had similarly good package ecosystems in the late 1990s, and I think any of them could have ended up as the dominant scripting language. Then Google chose Python as its main scripting language, invested hundreds of millions of dollars, and here we are. It's not as suitable as Matlab, R, or Julia for numerical work, but money made it good enough.
(Sort of like how Java and later JavaScript VMs came to dominate: you can always compensate for poor upfront design with enough after-the-fact money.)
I made that switch before I’d ever heard of Google, or Ruby for that matter. My experience was quite common at the time.
not for everything. For mobile apps is still very poor - even if you plan only for prototyping instead of distribution. Same for frontend and desktop. For desktop you do have pyqt and pyside but I would say experience is not as good - you would still better do at least doing UI in QML) and end user distribution still sux.
I wish python mobile story improve. Python 3.13 try to improve support for android and iOS and beeware also working on it. But right now ecosystem of pip wheels that build for mobile is very minimal.
Yes. Because C and C++ are never going to have a comparable package ecosystem, it almost makes sense for people to distribute such library projects as python packages simply because it handles all the packaging.
I really hope we are at the end game with poetry or uv. I can't take it anymore.
Arguably it could be a little easier to automatically start up a virtual environment if you call pip outside of one… but, I dunno, default behavior that papers over too many errors is not great. If they don’t get a hard error, confused users might become even more confused when they don’t learn they need to load a virtual environment to get things working.
It reads from a database. Generates embeddings. Writes it to a vector database.
- ~10X faster
- ~10X lower memory usage
All instructions on the Internet and even on the vector database documentation are in Python.
I also wrote a RAG pipeline in Go, using OpenSearch for hybrid search (full-text + semantic) and the OpenAI API. I reused OpenSearch because our product was already using it for other purposes, and it supports vector search.
For me, the hardest part was figuring out all the additional settings and knobs in OpenSearch to achieve around 90% successful retrieval, as well as determining the right prompt and various settings for the LLM. I've found that these settings can be very sensitive to the type of data you're applying RAG to. I'm not sure if there's a Python library that solves this out of the box without requiring manual tuning too
There are Python libraries that will simplify the task by giving a better structure to your problem. The knobs will be fewer and more high-level.
Sometimes you end up embedding chunks of javascript directly inside your python
For example the docs for Streamlit implementation of AgGrid contain this: https://staggrid-examples.streamlit.app/Advanced_config_and_...