The Alternative Implementation Problem
pointersgonewild.com
pointersgonewild.com
Let's say there's a piece of proprietary software for creating financial reports[1], which is using a weird binary format to store their documents. You want to make a free alternative, which can load and save these documents. The format is not fun to deal with, so you have a single function that loads the whole document into memory and one that dumps your data structures back to disk and entirely overwrites the file, but you operate purely on in-memory data while the software is running. What you don't know is that the proprietary software doesn't do this, it was developed in the days when users had very little RAM, so it only loads and saves the section the user is currently operating on, and knows how to modify the document in-place.
Then, the proprietary software introduces a way to add attachments to their documents. People keep adding stupidly large files, recordings of investor calls, scanned PDFs with hundreds of pages and so on. THe proprietary software loads documents one section at a time, so this works just fine. You, on the other hand, always deserialize the entire document at once, which suddenly becomes a problem when the documents are bigger than the users' available RAM memory. You now basically have to re-architect the entirety of your software, to support a change that took a week of a single developer's time for the main implementation.
[1] a purely hypothetical example, I've never actually worked in this space.
The author mentions this when they talk about the ease of implementing new features in an interpreted language vs. a compiled language.
I have only had to do 1 major rearchitecturing, and that was on the original implementation, which had some architectural assumptions that forced an exponential blowup. I'm wasn't working on the second implementation, but they started years after us, and managed to avoid our mistake from the beginning.
To take a public example, that is far more complicated than anything I have worked on, consider the problem of running Windows applications under Linux.
Linux is a completly different implementation of a kernel than NT, and doesn't even attempt to be compatible. However, running Windows applications does not require a rearchitecture. Just a few relativly generic Kernel features, and a userspace compatiblity layer. Not to say that Wine does not take a lot of effort to write and maintain; but not nearly as much effort as implementing Windows itself. And, it is running on top of a platform that never aspired to be Windows compatible. Wine does, however, suffer from the problem that the article points outs, in that it is perpetually behind Windows. Wine also exemplifies a second problem, which is that bugs in the primary implementation end up being relied upon, so in order to be bug-for-bug compatible, you need to first discover what all the bugs you need to implement are.
There's a good argument that implementing kernel-mode driver APIs would be a complete waste of time, after all, Wine is running on top of another operating system, which should be responsible for interfacing with hardware. However, what the Wine developers didn't foresee was the fact that some applications (mainly games) started requiring a driver to function as an anti-cheat feature.
Very interesting, what industry if you may discuss it a bit more?
[1] This is typically because the expected outputs are not easily known or specified in advance (e.g., interpolating a 3D wind cube to a higher resolution from lower resolution forecast data) and there isn't much if any experimental data, and collecting such data is expensive because it requires flying expensive aircraft around for long periods of time.
In fact this is one of the few times you can clearly illustrate tech debt to management. It just takes us longer than Acme to implement these features.
Yes. This is the classic problem with making Python faster. CPython started as a naive interpreter, one that just plods along doing what the code says without optimization. In CPython, everything is a dictionary and there's little real concurrency. This allows any code to patch anything else on the fly. Nobody does that much, but people scream if you try to take that out. So implementations which really compile Python have to be able to handle some thread suddenly modifying something out from under another thread.
Yet both ecosystems were able to come up with JIT implemenations that can handle the "reboot the world at any moment" use case.
Would you say that's also true of Smalltalk implementations?
As-far-as I can see those comments are unhelpful to the point of being evasive.
That's not an example of comparable complexity to "… Python ha[s] to be able to handle some thread suddenly modifying something out from under another thread".
> don’t go trying to create a subset of Python
I agree - a project that's marketed as "Python but with more X" is always going to struggle to compete with the canonical implementation. (Especially if X is speed - ultimately, if you're using a dynamically-typed language you probably don't care about execution speed).
However, alternative implementations are not always doomed to failure. MicroPython seems to be somewhat successful, despite supporting little more than Python 3.4 (10 years old). It's designed to run on microcontrollers so it's not competing with CPython - it's competing with other microcontroller programming environments.
OTOH, I'm sure the MicroPython maintainers get a lot of feature requests for more recent Python features. I once considered building an alternative implementation of Python that's lightweight and focused on embedding in applications - again, it wouldn't be competing with CPython, but with Lua. The number one feature request seemed to be "will it support NumPy?"
That sword cuts both ways.
The reason the competition in a field has converged on all the same features, all looking the same and acting the same is because they were all shaped by the same market forces: that's what the businesses wanted!
If a potential client looks at your product and it doesn't look like a duck, doesn't talk like a duck and doesn't walk like a duck, they're going to assume that it's not a duck.
I've pitched a fairly simple product (still iterating on it so not sharing details) that had some extra features (not AI) to a business, and the business eventually went with a more expensive competing product, with my contact at the place explaining "The stakeholders feel that your product is for a slightly different use-case."
Yes, my product did have all the features of the competition. The added feature was low-code extensibility for the backend API. The business interpreted the API-extending-mechanism as "Something only big companies would use", not "Something we can ignore if we don't use".
Now, fair enough, this is absolutely an outlier - in most cases extensibility is regarded favourably. But, in this edge-case the audience came away with the primary impression of "Great Development Tool", not "Great User Product"[1].
Humans still have this notion that a thing has a primary purpose and multiple secondary purposes, and they'll absolutely go with the product that has, as it's primary purpose, satisfying their need, even if some other product's secondary purpose also satisfies their need.
[1] And yes, this was a failure of the pitch. For future pitches, I'll tailor to the audience, highlighting their needs and ignoring anything that the product does which isn't in their list of needs.
By this logic, we all want AI. We're screaming out for Microsoft/Amazon/Google to make all their services AI-driven.
I'm sure some customers do want AI, but mostly it's their investors.
With AI, specifically, it's way too early to say that products with AI in it were shaped by market forces.
It takes years for the forces of the market to have an effect on what the product looks like:
1. Purchasers have to make poor purchases, which isn't known until years later.
2. Sellers have to get feedback from rejections to determine what to refine, and how, which also takes years for most products.
3. Sellers have to run out of money when ignoring the signals, which also takes years.
So, sure, maybe extra AI in $PRODUCT isn't wanted by the majority of people, but we won't actually know what forces the market for $PRODUCT is exerting until much later than 2024.
EDIT: web3 was so obviously unwanted, and yet it took about 4 years for that to reflect. It is not so clear about AI, so we can expect that to take longer to establish.
where someone is like 'okay we need our own internal version of this API'. the reasons vary but are like 'we don't trust people to use the official API correctly' (which, fine, but yours is less standard and worse-documented).
sometimes the answer is 'we need extra functionality' which, fine, but 1) don't wrap the entire API for gods sake, just add 3 functions, and 2) your codebase over time will end up 99% polyfills
point being if you are not using the defaults you are inflicting great misery on whoever inherits your codebase
For example, there is the ziggy-pydust library to write native modules for Python in Zig, and it certainly looks prettier than the usual Python.h import. Still, there is also .ffi to address functions directly that are not yet implemented but are available in Python.h.
If such an option is not available, one often wants to replace such a library and prefer the original one.
However, even this is in a way a wrapper (a native module), and sometimes it is better to use ctypes directly, for example, for the sake of speed of development.
It's missing a key ingredient though...
I see this as any other competing alternative to any kind of product. It's like saying Amazon failed because the didn't have a brick and mortar book store like people were used to.. obviously that's not the case
The reason all these jit-alts failed and were in a constant catch up, is because in practicality, most developers of x-language don't care that much about JIT.
Or more importantly, they care more about language features and interoperability than they care about JIT. This is why a product that "joins them" rather than competing wins. Because you are not offering more stability and/or interoperability.
Another way to phrase the same idea: a language is _a lot more_ than just compile speed.
Compile speed is very important, in the top 10 dimensions, for sure. Especially because improving compile speed increases the developer feedback loop clock speed which helps the core team improve all the other dimensions faster.
But there are still >30 other dimensions that are very important in a programming language.
Combine that with the fact these languages are often used to make CRUD websites where I/O is likely to be a bigger performance factor than CPU and the faster alternatives look a lot less appealing.
Maintainers of a language implementation would like to have maximum flexibility in designing and shipping new features that benefit their users. They don't want to be hamstrung by having to get multiple implementations to agree before they can get a feature out the door. As a case study, look at how glacially slow JavaScript evolution was for many years. And look at how comparatively fast TypeScript evolves.
At the same time, alternate implementations can be a signal of the robustness of your language ecosystem, so there is an upside too. And if the alternate implementation is really good (even if just for some subset of users with niche requirements), it can be a real value add for your ecosystem.
So if you're a language designer/maintainer, you might not be actively hostile to alternate implementations, but there are downsides. And, for the most part, the feedback you'll get from users will be towards encouraging you to ship new features and evolve the language. You won't get too many people asking you to slow down so that PyPy/IronRuby/LuaJIT/etc. can keep up.
For a language consumer, choosing which implementation to build on is often a choice where the biggest priority is safety and stability. No one wants to find themselves sitting on a million-line codebase that happens to subtlely rely on the behavioral quirks of some languishing alternate implementation created by a very bright PhD student who has since moved onto other projects. So there's a very strong positive feedback loop where users tend to go to the most-used implementation, which then encourages other users to go to that implementation, which encourages... The network effects are quite strong.
The end result is that unless there are quite strong forces pushing in the other direction, most languages tend towards a single canonical implementation. There's a good argument that that's a good thing: it means that almost all engineering effort put into implementing the language benefits all users instead of it being divided among a number of separate implementations. The downside, of course, is that the implementation can get stuck in a local maximum.
Adding a JIT to an existing language is a major undertaking, so the canonical implementation will have a high bar for acceptance. But you should probably still aim for that. Forking or rolling your own can provide freedom to do big things too, but that should often be regarded as temporary.
If your goal is to show what you can do, it probably won't go far. If your goal is to make something better for its good, you'll learn to work within others constraints.
All the people that preach about single project, contributing only to canonical project and having no alternative implementations need to check their egos, learn about existance of multiple approaches and why competition is better than monopoly
As I see it, article shows how Python, Lua and Ruby failed many people by choosing this approach - causing thousands of devs and millions of users to have slower development and software. Not because it's not possible - but because administratively there's no insentive to do that
Then tell me: why is competition better than monopoly for these [!] people?
I don't think it's surprising that micropython/circuitpython have had more success, in many ways they are a port to an environment where the existing libraries wouldn't be used, so they can at some level build their own ecosystem.
Sure, but that doesn’t contradict what I said. Numpy is surprisingly old, but in 2010 (my impression was that) supporting it was in no way table stakes for an implementation of Python. Hell, Jython was still occasionally taken seriously then, and Microsoft still pretended they cared about IronPython.
> I would suggest the C API is just as much a part of Python as the standard library
As things stand, yes, no question.
The problem is, it was never designed that way. In particular, it cannot (or at least is not designed to) support any other memory management strategy but CPython’s, thus efforts like HPy[1]. For example, while the language docs admonish you not to depend on __del__ running at any particular moment, if we consider the Python/C API a part of the language, then the language has to behave as though it uses naïve reference counting and occasionally runs a cycle collector. And that’s the easy part—think about how deeply the GIL is rooted there.
That is not good design for something that’s part of the language, as it effectively is. It was never intended to be and never received commensurate amounts of care. It just turned out that way—and ain’t that a sad thing to admit.
If anything Chrome is an exception to the article's "Alternative Implementation Problem"; what started as Konquerer, an alternative browser, eventually mutated into the most common desktop browser currently in use.
More so, you are also limited in terms of popular libraries versions, and it is much more cumbersome because of versioning hell itself.
Microsoft is still actively maintaining MSVC
Also another interesting tidy bit, all those compiler vendors that leech from clang/LLVM hardly contribute to upstream in regards to ISO C++ compliance, hence why nowadays clang is lagging behind versus its initial velocity.
Insightful, as I witnessed the problem caused by TiDB's compatibility with MySQL.
Without Google pushing Kotlin no matter what on Android, Kotlin would be yet another JVM guest language.
Nothing can be further from the truth. A small minority, in any language, not just Python, will use the most recent version. In any language community, the most used version lags several iterations behind the most recently released. In a decade and a half of working with Python, changing plenty of companies and positions, I've never been in a situation where I was using the newest version right after it was released. Not for the "production" environment anyways.
The reason PyPy is not popular is... if I had to guess that Python is a first language for many programmers (i.e. the its audience is mostly not very knowledgeable in general, and will struggle to install a Python, installing a special kind of Python will be too overwhelming for them). Python is also a popular language with people who aren't professional programmers, eg. researchers, statisticians. In many places where Python is used its speed makes no difference, even if its used by professional programmers (eg. various kinds of automation). So, there's no particular reason to choose PyPy.
To top this: Python is a common language for OS scripting / automation, where interpreter starting time may be more important than the performance of the script.
PyPy mostly doesn't support native Python modules, but even when it does, it offers no benefits in so doing.
Finally, there's a low-hanging fruit for any Python programmer who makes Python packages: just run them through Cython. This alone gives a significant boost to speed (and some memory savings). Virtually nobody does this. Which is, imo, an indication that speed isn't important. (Also, fighting Python infrastructure and trying to understand the packaging process isn't for the faint of heart, which underscores my previous point of general low competency on the part of Python users).
----
Bottom line: PyPy doesn't offer anything of value to the overwhelming majority of Python users. It would've been a headache to work with even where there might be potential benefits. So... it's not used. Python's release schedule is laughably bad, and new features are laughably bad... well, worthless for the most part... but they aren't the reason PyPy isn't popular.
I only use Python for portable OS scripting tasks because of that, if they don't need to be portable, it is either UNIX shell/Perl, or Powershell.
To what extent is this an indictment of capitalism and/or free market ideology?
The linked Thiel speech video is interesting. It says that the aim of a capitalist is not to compete, but to create a monopoly. This is starkly against the interests of the consumer, and we see the results of that today with the enshittification of modern tech.
Are we at a stage where the capitalist elites themselves are freely admitting that the system works for them and against the general public?
There exist lots of kinds of monopolies; only for some of them a state is necessary to enforce them.
Citation needed.
One of the problems that arise from things that try to be the same is that by definition the value of switching can't be that great. So it doesn't really take much cost to derail the process. Things that are radically different may have significant advantages that make it worthwhile. If you are going to make something "the same" it needs to be very very the same. Such things do sometimes happen, e.g., the competitive for-pay JVM space.
This is true for things like programming languages and runtimes, which are always intrinsically huge. It doesn't apply to which brand of bottled water you may prefer because those are quite interchangeable for no cost.
This is one of the reasons business people are always interested in the features that differentiate their product. If you want someone to pay the cost to switch you need to bring them something more than just "it works as well", in general.
I think this is almost what the article's saying, but not quite. The point is that even if you do make something exactly the same but better than a canonical product, you're then trapped into playing catch-up with all developments in the canonical product, and have little to no directional control.
> It doesn't apply to which brand of bottled water you may prefer because those are quite interchangeable for no cost.
You might be right, but I can see analagous arguments applying even to the bottled water space. Imagine I create something that is "Evian but cheaper". When Evian adds a feature such as reducing the amount of plastic per bottle cap, or improving its mineral content, I have to match it in order to keep growing.
I've been grappling with similar questions for a while. I've been focusing less on the nominal replacements like the author did, and looking more at our general inability to create a programming language that is "like" another language, but just generally improved. C++ is almost literally the last example that really took off. Python 3 was something like what I'm talking about, and while Python survived Python 3, I'm not sure I can call it a success.
But we could really use "like X but modern" in quite a few places. I'd love to see a modern dynamic scripting language that is a lot like the current ones, but, for instance, was designed from day to work with threading somehow. In this case I'm not referencing async versus thread debates, I'm not talking about how it gets exposed to the users, I'm talking about the difficult that 10-20 year old scripting languages still have to this day using multiple processors in any reasonable way. And maybe we've learned we can get most of the benefits of dynamic scripting languages without quite being as dynamic as the current crop is, so maybe instead of a 10-40x slowdown we could be looking at another 2 or 3 speed increase over the current crop with only minimal feature loss. And a few other things.
Various such languages arguably exist, but they are starved of oxygen.
As for the bottled water point, if you look with sufficient detail you can eventually figure out why this Evian bottle is better for you than that one. No two things are identical. But if you can't how the differences between two brands of bottled water and two entire language implementations are a sufficient difference in quantity to be a difference in quality... several times over, honestly... I don't really know what to say.
It's hard to be so much better that old users will want to switch. But once you are that much better, you steal all the users of the competition. So, it could be worth a try, if you believe you can be that much better. (The original UNIX and a bunch of stuff related to it, s.a. compilers, editors etc. many of which found their reimplementations in Linux).
But, not only that. Some projects live well in the shadow of the "canonical" twin. Take Cassandra and Scylla. The later is alive and well, even though may not be as popular as the former.
Speaking of Python. Anaconda Python is an alternative distribution of python.org Python. They aren't going to steal the spotlight from the "canonical" Python any time soon, but they won't go away either because of some compelling features their competition doesn't offer.
Honestly. I don't believe there's a pattern. The deeper problem here that I see is the lack of competition. The fact that every company is trying to build something that isn't an alternative, but a different thing altogether leads to all the different variants of what would've been essentially the same thing being garbage: because nobody has the capacity for substantial development and testing.
We have hundreds of thousands custom-made e-commerce shops, all of which suck. Having e-commerce shop makers compete in the same category rather than each in the category of their own would've created a high standard for user experience of shopping online.