The future of 32-bit support in the kernel
lwn.net
lwn.net
It's not a well argumented thought, just a nagging feeling.
Maybe we need a simple posix os that would run on a simple open dedicated hardware that can be comprehended by a small group of human beings. A system that would allow communication, simple media processing and productivity.
These days it feels like we are at a tipping point for open computing. It feels like being a frog in hot water.
Open source is one thing, but open hardware - that’s what we really need. And not just a framework laptop or a system76 machine. I mean a standard 64-bit open source motherboard, peripherals, etc that aren’t locked down with binary blobs.
If that comes to pass we will want software that run on earlier nodes and 32bit hardware.
I doubt anyone here has a clean enough room.
Jordan Peterson has entered the building...
We'll likely never have "affordable" photolithography, but electron beam lithography will become obtainable in my lifetime (and already is, DIY, to some degree.)
However, making at home a useful microcontroller or FPGA would require not only an electron-beam lithography machine, but also a ion-implantation machine, a diffusion furnace, a plasma-etch machine, a sputtering machine and a lot of other chemical equipment and measurement instruments.
All the equipment would have to be enclosed in a sealed room, with completely automated operation.
A miniature mask-less single-wafer processing fab could be made at a cost several orders of magnitude less than a real semiconductor fab, but the cost would still be of many millions of $.
With such a miniature fab, one might need a few weeks to produce a batch of IC's worth maybe $1000, so the cost of the equipment will never be recovered, which is why nobody does such a thing for commercial purposes.
In order to have distributed semiconductor fabs serving small communities around them, instead of having only a couple of fabs for the entire planet, one would need a revolution in the fabrication of the semiconductor manufacturing equipment itself, like SpaceX has done for rockets.
Only if the semiconductor manufacturing equipment would be the result of a completely automated mass production, which would reduce its cost by 2 or 3 orders of magnitude, affordable small-scale but state-of-the-art fabs would be possible.
But such an evolution is contrary to everything that the big companies have done during the last 30 years, during which all smaller competitors have been eliminated, the production has become concentrated in quasi-monopolies and for the non-consumer products the companies now offer every year more and more expensive models, which are increasingly affordable only for other big companies and not for individuals or small businesses.
some revolution. still not even on the moon yet
University nanofabs have all of these things today. https://cores.research.asu.edu/nanofab/
> but the cost would still be of many millions of $.
A single set of this equipment is only singular millions today commercially.
Using something like this for prototyping/characterization or small-scale analog tasks is where the real win is.
It is weird that they do not have any ion implantation machine, because there are devices that are impossible to make without it. Even for simple MOS transistors, I am not aware of any other method for controlling the threshold voltage with enough precision. Perhaps whenever they need ion implantation they send the wafers to an external fab, with which they have a contract, to be done there.
Still, I find it hard to believe that all the equipment that they have costs less than 10 million $, unless it is bought second hand. There is indeed a market for slightly obsolete semiconductor manufacturing equipment, which has been replaced in some first tier fabs and now it is available at significant discounts for those who are content with it.
Maybe?
Another point of view might be that in a few weeks you could produce a batch of ICs you can actually trust, that would be several orders of magnitude more valuable than the $1000 worth of equivalents from the untrusted global supply chain.
"The principle of evidence-based trust was at work in our decision to implement Precursor’s brain as an SoC on an FPGA, which means you can compile your CPU from design source and verify for yourself that Precursor contains no hidden instructions or other backdoors. Accomplishing the equivalent level of inspection on a piece of hardwired silicon would be…a rather expensive proposition. Precursor’s mainboard was designed for easy inspection as well, and even its LCD and keyboard were chosen specifically because they facilitate verification of proper construction with minimal equipment."
See also: https://betrusted.io
Wafer machines from the 1970s could be fairly cheap today, if there were sufficient demand for chips from the 1970s (~1MHz, no power states, 16 bit if you’re lucky, etc), but that trend would have to stop and reverse significantly for affordable wafer factories for modern hardware to be a thing.
Let’s hope some of that trickles down to consumer hardware.
The problem here is scale. Having fully-open hardware is neat, but then you end up with something like that Blackbird PowerPC thing which costs thousands of dollars to have the performance of a PC that costs hundreds of dollars. Which means that only purists buy it, which prevents economies of scale and prices out anyone who isn't rich.
Whereas what you actually need is for people to be able to run open code on obtainium hardware. This is why Linux won and proprietary Unix lost in servers.
That might be achievable at the low end with purpose-built open hardware, because then the hardware is simple and cheap and can reach scale because it's a good buy even for people who don't care if it's open or not.
But for the mid-range and high end, what we probably need is a project to pick whichever chip is the most popular and spend the resources to reverse engineer it so we can run open code on the hardware which is already in everybody's hands. Which makes it easier to do it again, because the second time it's not reverse engineering every component of the device, it's noticing that v4 is just v3 with a minor update or the third most popular device shares 80% of its hardware with the most popular device so adding it is only 20% as much work as the first one. Which is how Linux did it on servers and desktops.
is this even doable?
Could Amazon or Facebook do this if they just wanted to, e.g. to help break the hold of their competitors on markets they care about? Absolutely.
Could some hobbyist do it? Not on their own, but if you do part of it and someone else does part of it, the whole thing gets done. See e.g. Asahi Linux.
also this is what happen to prusa, everyone just take the design and outsource the manufacture to somewhere in china which is fine but if everybody doing that, there is no fund to develop next iteration of product (someone has to foot the bill)
and there is not enough sadly, we live in reality after all
Which means, "open" has nothing to do with openness. What you want is standardization and commoditization.
There are practically no x86 hardware that require model-specific custom images to boot. There are practically no non-x86 hardware that don't require model-specific custom images to boot. ARM made perceptible amount of efforts in that segment with Arm SystemReady Compliance Program, which absolutely nobody in any serious businesses cares about, and it only concern ARM machines even if it worked.
IMO, one of problems in efforts going in from software side is over-bloated nature of desktop software stacks and bad experiences widely had with UEFI. They aren't going to upgrade RAM to adopt overbloated software that are bigger than the application itself just because that is the new standard.
Open hardware you can buy now: https://www.crowdsupply.com/sutajio-kosagi/precursor
The open OS that runs on it: https://betrusted.io/xous-book/
A secret/credential manager built on top of the open hardware and open software: https://betrusted.io
His blog section about it: https://www.bunniestudios.com/blog/category/betrusted/precur...
"The principle of evidence-based trust was at work in our decision to implement Precursor’s brain as an SoC on an FPGA, which means you can compile your CPU from design source and verify for yourself that Precursor contains no hidden instructions or other backdoors. Accomplishing the equivalent level of inspection on a piece of hardwired silicon would be…a rather expensive proposition. Precursor’s mainboard was designed for easy inspection as well, and even its LCD and keyboard were chosen specifically because they facilitate verification of proper construction with minimal equipment."
This needs money. It is always going to have to pay the costs of being niche, lower performance, and cloneable, so someone has to persuade people to pay for that. Hardware is just fundamentally different. And that's before you get into IP licensing corner cases.
I think people here are misunderstanding just how "weird" and hacky trying to run an OS like linux on those devices really is.
Not having an MMU puts you more into the territory of DOS than UNIX. There is FreeDOS but I'm pretty sure it's x86-only.
The one thing different to a regular Linux was that a crash of a program was not "drop into debugger" but "device reboots or halts". That part I don't miss at all.
How do multiple processes actually work, though? Is every executable position-independent? Does the kernel provide the base address(es) in register(s) as part of vfork? Do process heaps have to be constrained so they don't get interleaved?
Executables in a no-MMU environment can also share the same code/read-only segments between many processees, the same way shared libraries can, to save memory and, if run-time relocation is used, to reduce that.
The original design of UNIX ran on machines without an MMU, and they had fork(). Andrew Tanenbaum's classic book which comes with Minix for teaching OS design explains how to fork() without an MMU, as Minix runs on machines without one.
For spawning processes, vfork()+execve() and posix_spawn() are much faster than fork()+execve() from a large process in no-MMU environments though, and almost everything runs fine with vfork() instead of fork(), or threads. So no-MMU Linux provides only vfork(), clone() and pthread_create(), not fork().
[1]: https://maskray.me/blog/2024-02-20-mmu-less-systems-and-fdpi...
[2]: https://popovicu.com/posts/789-kb-linux-without-mmu-riscv/
[3]: https://www.kernel.org/doc/Documentation/nommu-mmap.txt
[4]: https://github.com/kraj/uClibc/blob/ca1c74d67dd115d059a87515...
I spent close to ten years working closely with uClinux (a long time ago). I implemented the shared library support for the m68k. Last I looked, gcc still included my additions for this. This allowed execute in place for both executables and shared libraries -- a real space saver. Another guy on the team managed to squeeze the Linux kernel, a reasonable user space and a full IP/SEC implementation into a unit with 1Mb of flash and 4Mb of RAM which was pretty amazing at the time (we didn't think it was even possible). Better still, from power on to login prompt was well under two seconds.
The original UNIX also did not have the virtual memory as we know it today – page cache, dynamic I/O buffering, memory mapped files (mmap(2)), shared memory etc.
They all require a functioning MMU, without which the functionality would be severely restricted (but not entirely impossible).
The no-MMU version of Linux has all of those features except that memory-mapped files (mmap) are limited. These features are the same as in MMU Linux: page cache, dynamic I/O buffering, shared memory. No-MMU Linux also supports other modern memory-related features, like tmpfs, futexes. I think it even supoprts io_uring.
mmap is supported in no MMU Linux with limitations documented here: https://docs.kernel.org/admin-guide/mm/nommu-mmap.html For example, files in ROM can be mapped read-only.
Access to a page that is not resident in memory results in a trap (an interrupt), which is handled by the MMU – the CPU has no ability to do it by itself. Which is the whole purpose of the MMU and was a major innovation of BSD 4 (a complete VMM overhaul).
But three out of those four features: page cache, dynamic I/O buffering and shared memory between processes, do not require that kind of VMM subsystem, and memory-mapped files don't require it for some kinds of files.
I've worked on the Linux kernel and at one time understood it's mm intimately (I'm mentioned in kernel/futex/core.c).
I've also worked on uClinux (no-MMU) systems, where the Linux mm behaves differently to produce similar behaviours.
I found most userspace C code and well-known CLI software on Linux and nearly all drivers, networking features, storage, high-performance I/O, graphics, futex, etc run just as well on uClinux without source changes, as long as there's enough memory, with some more required because uClinux suffers from a lot more memory fragmentation due to needing physically-contiguous allocations).
This makes no-MMU Linux a lot more useful and versatile than alternative OSes like Zephyr for similar devices, but the limitations and unpredictable memory fragmentation issues make it a lot less useful than Linux with an MMU, even if you have exactly the same RAM and no security or bug concerns.
I'd always recommend an MMU now, even if it's technically possible for most code to run without one.
With the right filesystem (certain kinds of read-only), the code (text segment) can even be mapped directly, and no loading into RAM need occur at all.
These approaches saves memory even on regular MMU platforms.
"Originally, fork() didn't do copy on write. Since this made fork() expensive, and fork() was often used to spawn new processes (so often was immediately followed by exec()), an optimized version of fork() appeared: vfork() which shared the memory between parent and child. In those implementations of vfork() the parent would be suspended until the child exec()'ed or _exit()'ed, thus relinquishing the parent's memory. Later, fork() was optimized to do copy on write, making copies of memory pages only when they started differing between parent and child. vfork() later saw renewed interest in ports to !MMU systems (e.g: if you have an ADSL router, it probably runs Linux on a !MMU MIPS CPU), which couldn't do the COW optimization, and moreover could not support fork()'ed processes efficiently.
Other source of inefficiencies in fork() is that it initially duplicates the address space (and page tables) of the parent, which may make running short programs from huge programs relatively slow, or may make the OS deny a fork() thinking there may not be enough memory for it (to workaround this one, you could increase your swap space, or change your OS's memory overcommit settings). As an anecdote, Java 7 uses vfork()/posix_spawn() to avoid these problems.
On the other hand, fork() makes creating several instances of a same process very efficient: e.g: a web server may have several identical processes serving different clients. Other platforms favour threads, because the cost of spawning a different process is much bigger than the cost of duplicating the current process, which can be just a little bigger than that of spawning a new thread. Which is unfortunate, since shared-everything threads are a magnet for errors."
https://stackoverflow.com/questions/8292217/why-fork-works-t...
FreeBSD is dumping 32 bit:
https://www.osnews.com/story/138578/freebsd-15-16-to-end-sup...
OpenBSD has this quote:
>...most i386 hardware, only easy and critical security fixes are backported to i386
I tend to think that means 32bit on at least x86 days are numbered.
https://www.openbsd.org/i386.html
I think DragonflyBSD never supported 32bit
For 32bit, I guess NetBSD may eventually be the only game in town.
The industry has a lot of experience doing so.
In parallel, the old hardware is still supported, just not by the newest Linux Kernel. Which should be fine anyway because either you are not changing anything on that system anyway or you have your whole tool stack available to just patch it yourself.
But the benefit would be a easier and smaller linux kernel which would probably benefit a lot more people.
Also if our society is no longer able to produce chips in a commercial way and we loose all the experience people have, we are probably having a lot bigger issues as a whole society.
But I don't want to deny that it would be nice to have the simplest way of making a small microcontroller yourself (doesn't has to be fast or super easy just doable) would be very cool and could already solve a lot of issues if we would need to restart society from wikipedia.
Think about it like CreateProcess() on Windows. Windows is another operating system which doesn't support fork(). (Cygwin did unholy things to make it work anyway, IIRC.)
Note ELKS is not Linux.
There's also Fuzix.
Linux remains open source, extendable, and someone would most likely maintain these ripped out modules. Just not at the expense of the singular maintainer of the subsystem inside the kernel.
Linux's master branch is actually called master. Not that it really matters either way (hopefully most people have realised by now that it was never really 'non-inclusive' to normal people) but pays to be accurate.
In this case, out of dozens of ways the word is used (others being like ‘masters degree’ or ones that pretty closely match Git’s usage like ‘master recording’ in music), one possible one is ‘slavemaster’, so some started to assert we have to protect people from possibly making the association.
The relation between a human and a computer is very much that of a master and a slave. I provide the resources, you do what I say, including self-destruction.
> Not that it really matters either way
But you still decided to make it your personal battle to make a comment remind people that the evil inclusive people did a no no by forcing you to rename your branches from master to main, yes.
>pays to be accurate.
No, it doesn't. Source: you knew exactly what I was talking about when I said main. You also would have known what I was talking about had I said trunk, master, develop or latest.
That isn't necessarily the case. You can have memory protection without a MMU - for instance, most ARM Cortex-M parts have a MPU which can be used to restrict a thread's access to memory ranges or to hardware. What it doesn't get you is memory remapping, which is necessary for features like virtual memory.
If you want a hardware architecture you can easily comprehend - and even build your own implementation of! - that's something which RISC-V handles much better than ARM ever did, nommu or otherwise.
At some point, it might start making sense to level up the OS to (nommu) Linux on these devices. When the applications get complex enough, people find themselves wanting a full blown network stack, full featured storage/file systems that are aligned with non-embedded systems, regular shells, POSIX userland, etc.
All of the architectures I have in mind are 32 bit and "nommu"[1]: Cortex-R52/F, Infineon TriCore, Renesas RH850, NXP Power e200. Then you have RISC-V MCU Cambrian Explosion underway.
I qualify all this with mays and mights: it hasn't happened yet. I'm just careful not to discount the possibly of a <50 mAh RP Pico 3 booting uLinux, running python and serving web pages being a big hit.
[1] They all have various "partition" schemes to isolate banks of RAM for security, reliability, etc., but real MMUs are not offered.
Linux is too unorthogonal for them.
When you have a fleet of embedded devices you want pre-compiled disk images, repeatable builds, read only filesystems, immutable state (apart from small and well controlled partitions), easy atomic updates, relatively small updates (often devices are at the other end of a very slow cell connection) and a very clear picture of what is running on every device in your fleet.
Linux can do all that, but it's not the paradigm that most distros take. Pretty much the entire Linux world is built around mutable systems which update in place. So you're left to manage it yourself. If you want to do it yourself, you end up in the hacky fragile world of Yocto.
Compared to that, using an RTOS or application framework like Zephyr is fairly easy - at the expense of app development time, you just need to worry about getting a fairly small compiled binary onto your device.
I do agree that there's some really powerful parts available which would benefit from the shared drivers and consistent syscalls a standardised operating system offers. But building and maintaining a Linux system for any part isn't a simple undertaking either - and so the complexity of that needs to be considered in total development time.
It's a 5 gallon pail of compute which is all used up in OS overhead so you can do a cup of work.
If the job were that small that it fits in the remainder, then you could and should have just used 1 cent hardware instead of 1 dollar hardware.
As a reality check: MicroPython can run in 16 KB of RAM; a typical development board has 192 KB. µCLinux requires at least 4 - 8 MB of RAM just to boot up, and recommends 32 MB for a "serious product" [1].
> Linux has solid network, WiFi, Bluetooth stacks and a rich box of tools that might be very nice to tap into without requiring something as big as an RPi.
I would absolutely not count on any of those tools being available and functional in a µCLinux environment.
[1]: https://www.emcraft.com/imxrt1050-evk-board/what-is-minimal-...
My point exactly. There's currently a hole between ten-cent MCUs requiring RTOS and 5$+ RPi that can run Linux. Taking out nommu from the kernel would make any 64MB "super-MCU" a non-starter.
I'm not convinced that's true. All of the microcontroller tooling I've seen has been built around RTOS development; I've never seen anything comparable for µCLinux.
What's missing? What would it take to make a plain RTOS that's as easy to develop on/for/with as Linux?
That's all worth it to have an application processor that can run your Python/Java app. It's probably worth it to have a consistent operating system across multiple devices.
Would you have many of those benefits if you were using Linux on a micro though? I can't imagine much 3rd party software would work reliably given the tiny amount of RAM. You'd basically just be using it as a task scheduler and wrapper over drivers. You could get most of the benefits by using an RTOS with a memory allocator.
Two things standout to me: (1) It was written in 2006. RISC V was not released until 2010 (so says Google). I guess it was ported from x86? (2) Russ Cox is one of the contacts listed on MIT homepage. That guy's digital footprints are enormous.
And given the target market (even the biggest Zephyr targets have addressable RAM in the megabytes, not gigabytes), there's no self-hosting notion. You build on a big system and flash to your target, always. So there's no single filesystem image (though it supports a somewhat limited filesystem layer for the external storage these devices tend to see) containing "programs" to run (even though there's a ELF-like runtime linker to use if you want it).
If you want it to look like Linux: no, that's not what it's for. If you want to target a Cortex-M or ESP32 or whatever on which Linux won't run, it gives you the tools you expect to see.
This is a young mans' game, and I am very much not.
Let's look at the lowest end chip in the discussion. Almost certainly the SAM9x60.... it is a $5 ARMv5 MMU chip supporting DDR2/LPDDR/DDR3/LPDDR3/PSRAM, a variety of embedded RAM and 'old desktop RAM' and mobile RAM.
Yes it's 32-bit but at 600MHz and GBits of RAM support. But you can seriously mass produce a computer under $10 with the chip (so long as you support 4-layer PCBs that can breakout the 0.75mm pitch BGA). As in, the reference design with DDR2 RAM is a 4-layer design.
There are a few Rockchips and such that are (rather large) TQFP that are arguably easier. But since DDR RAM is BGA I think it's safe to assume BGA level PCB layout as a point of simplicity.
---------
Everything smaller than this category of 32-bit / ARMv5 chips (be it Microchip SAM9x60, or competing Rockchips or AllWinner) is a microcontroller wholly unsuitable for running Linux as we know it.
If you cannot reach 64MBs of RAM, Linux is simply unusable. Even for embedded purposes. You really should be using like FreeRTOS or something else at that point.
---------
Linux drawing the line at 64MB hardware built within the last 20 years is.... reasonable? Maybe too reasonable. I mean I love the fact that the SAM9x60 is still usable for modern and new designs but somewhere you have to draw the line.
ARMv5 is too old to compile even like Node.js. I'm serious when I say this stuff is old. It's an environment already alien to typical Linux users.
A $1 Linux capable ARM: https://www.eevblog.com/forum/microcontrollers/the-$1-linux-...
I'd expect that there were even cheaper processors now since it's eight years later.
Fully open hardware, with mainline Linux open source drivers. It's hard to beat SAM9x60 in openness, documentation and overall usability. It's specs are weaker but keeping up with mainline Linux is very, very relevant. Especially in this discussion
Most likely I won't be around this realm when that takes shape, but I predict the GNU/Linux explosion replacing UNIX was only a phase in computing history, eventually when everyone responsible for its success fades away, other agendas will take over.
It is no accident that the alternatives I mention, are all based on copyleft licenses.
Simple and POSIX would be a BSD like NetBSD or OpenBSD.
This is why I gravitated to Plan 9. Overall a better design for a networked world and can be understood by a single developer. People can and have maintained their own forks. Its very simple, small and cross platform was baked in from day one. 9P makes everything into a IO socket organized as a tree of names objects. Thankfully it's not POSIX which IMO is not worth dragging along for decades. You can port Unix things with libraries. It also abandons the typewriter terminal and instead uses graphics. A fork, 9front, is not abandoning 32 bit any time soon AFIK. I netboot an older Industrial computer that is a 400MHz Geode (32 bit x86) with 128 MB RAM and it runs 9front just fine.
Its not perfect and lacks features but that stands to reason for any niche OS without a large community. Figure out what is missing for you and work on fixing it - patches welcome.
That was part of the plan for Minix 3.
Clean separation in a microkernel, simple enough for teaching students, but robust.
But Intel used it and gave nothing back, and AST retired. :-(
It will be relegated to the computing dustbin like non-8-bit bytes and EBCDIC.
Main-core computing is vastly more homogenous than when I was born almost 50 years ago. I guess that's a natural progression for technology.
So it's far more pervasive than people think, and will likely be in the picture for decades to come.
Of course they chose to integrate JavaScript so that's less likely now.
[1] Ok I admit, not trivially when it comes to unpaired surrogates, BOMs, endian detection, and probably a dozen other edge and corner cases I don't even know about. But you can offload the work to pretty well-understood and trouble-free library calls.
Most Unix syscalls use C-style strings, which are a string of 8-bit bytes terminated with a zero byte. With many (most?) character encodings you can continue to present string data to syscalls in the same way, since they often also reserved a byte value of zero for the same purpose. Even some multi-byte encodings would work if they chose to avoid using 0-value bytes for this reason.
UTF-16LE/BE (and UTF-32 for that matter) chose not to allow for this, and the result is that if you want UTF-16 support in your existing C-string-based syscalls you need to make a second copy of every syscall which supports strings in your UTF-16 type of choice.
That's completely wrong. If a syscall (or a function) expects text in encoding A, you should not be sending it in encoding B because it would be interpreted incorrectly, or even worse, this would become a vulnerability.
For every function, encoding must be specified as are specified the types of arguments, constraints and ownership rules. Sadly many open source libraries do not do it. How are you supposed to call a function when you don't know the expected encoding?
Also, it is better to send a pointer and a length of the string rather than potentially infinitely search for a zero byte.
> and the result is that if you want UTF-16 support in your existing C-string-based syscalls
There is no need to support multiple encodings, it only makes things complicated. The simplest solution would be to use UTF-8 for all kernel facilities as a standard.
For example, it would be better if open() syscall required valid UTF-8 string for a file name. This would leave no possibility for displaying file names as question marks.
Unfortunately some text encodings (UTF-16 among them) use nuls for codepoints other than U+00. In fact UTF-16 will use nuls for every character before U+100, in other words all of ASCII and Latin-1. Therefore you can't just support _all_ text encodings for filenames on these OSes, unless the OS provides a second syscall for it (this is what Windows did since they wanted to use UTF-16LE across the board).
I've only mentioned syscalls in this, in truth it extends all through the C stdlib which everything ends up using in some way as well.
It is good for an implementation to enforce this at some level, sure. MacOS has proved features like case insensitivity and unicode normalization can be integrated with Unix filename APIs.
And you should be using one specified encoding for file names if you want them to be displayed correctly in all applications. It would be inconvenient if different applications stored file names in different encodings.
For the same reason, encoding should be specified in libraries documentation for all functions accepting or returning strings.
UTF-16 is annoying, but it's far from the biggest design failure in Unicode.
UTF-32 is the worst of all worlds. UTF-16 has the teeny tiny advantage that pure Chinese text takes a bit less space in UTF-16 than UTF-8 (typically irrelevant because that advantage is outweighed by the fact that the markup surrounding the text takes more space). UTF-8 is the best option for pretty much everything.
As a consequence, never use UTF-32, only use UTF-16 where necessary due to backwards compatibility, always use UTF-8 where possible.
There's also the problem that grapheme cluster boundaries change over time. Unicode has become a true mess.
Not really. Unicode is still fundamentally based off of the codepoints, which go from 0 to 2^16 + 2^20, and all of the algorithms of Unicode properties operate on these codepoints. It's just that Unicode has left open a gap of codepoints so that the upper 2^20 codepoints can be encoded in UTF-16 without risk of confusion of other UCS-2 text.
The expansion of Unicode beyond the BMP was designed to facilitate an upgrade compatibility path from UCS-2 systems, but it is extremely incorrect to somehow equate Unicode with UTF-16.
Then there is also the issue that technically there is no such thing as UTF-16, instead you need to distinguish UTF-16LE and UTF-16BE. Even though approximately no one uses the latter we still can't ignore it and have to prepend documents and strings with byte order markers (another wasted pair of code points for the sake of an encoding issue) which mean you can't even trivially concatenate them anymore.
Meanwhile UTF-8 is backwards compatible with ASCII, byte order independent, has tons of useful properties and didn't require any Unicode code point assignments to achieve that.
The only reason we have UTF-16 is because early adopters of Unicode bet on UCS-2 and were too cheap to correct their mistake properly when it became clear that two bytes wasn't going to be enough. It's a dirty hack to cover up a mistake that should have never existed.
That's a strange way to characterize years of backwards compatibility to deal with
https://devblogs.microsoft.com/oldnewthing/20190830-00/?p=10...
I disagree you could just "easily" shove it into the "A" version of functions. Functions that accept UTF-8 could accept ASCII, but you can't just change the semantics of existing functions that emit text because it would blow up backwards compatibility. In a sense it is covariant but not contravariant.
And now, after you've gone through all of this effort: what was the actual payoff? And at what cost if maintaining compatibility with the other representations?
Also ISO 8601 (YYYY-MM-DD) should be the default date format.
I have a relatively large array of uint16_t with highly repetitive (low entropy) data. I want to serialize that to disk, without wasting a lot of space. I run compress2 from zlib on the data when serializinsg it, and decompress it when deserializing. However, these files make sense to use between machines, so I have defined the file format to use compressed little endian 16-bit unsigned ints. Therefore, if you ever want to run this code on a big-endian machine, you need to add some code to first flip the bytes around before compressing, then flipping them back after decompressing.
You're right that when your code is iterating through data byte for byte, you can write it in an endian-agnostic way and let the optimizer take care of recognizing that your shifts and ORs can be replaced with a memcpy on little-endian systems. But it's not always that simple.
I wish the same applied to written numbers in LTR scripts. Arithmetic operations would be a lot easier to do that way on paper or even mentally. I also wish that the world would settle on a sane date-time format like the ISO 8601 or RFC 3339 (both of which would reverse if my first wish is also granted).
> It will be relegated to the computing dustbin like non-8-bit bytes and EBCDIC.
I never really understood those non-8-bit bytes, especially the 7 bit byte. If you consider the multiplexer and demux/decoder circuits that are used heavily in CPUs, FPGAs and custom digital circuits, the only number that really makes sense is 8. It's what you get for a 3 bit selector code. The other nearby values being 4 and 16. Why did they go for 7 bits instead of 8? I assume that it was a design choice made long before I was even born. Does anybody know the rationale?
IIRC, in most countries the native format is D-M-Y (with varying separators), but some Asian countries use Y-M-D. Since those formats are easy to distinguish, that's no problem. That's why Y-M-D is spreading in Europe for official or technical documents.
There's mainly one country which messes things up...
If I'm writing a document for human consumption then why would I expect the dates to be sortable by a naive string sorting algorithm?
On the other hand, if it's data for computer consumption then just skip the complicated serialisation completely and dump the Unix timestamp as a decimal. Any modern data format would include the ability to label that as a timestamp data type. If you really want to be able to "read" the data file then just include another column with a human-formatted timestamp, but I can't imagine why in 2025 I would be manually reading through a data file like some ancient mathematician using a printed table of logarithms.
If you're naming a document for human consumption, having the files sorted by date easily without relying on modification date (which is changed by fixing a typo/etc...) is pretty neat
Its a serialization and machine communication format. And that makes me sad. Because YYYY-MM-DD is a great format, without a good name.
> NOTE: ISO 8601 defines date and time separated by "T". Applications using this syntax may choose, for the sake of readability, to specify a full-date and full-time separated by (say) a space character.
For the representation of text of an alphabetic language, you need to hit 6 bits if your script doesn't have case and 7 bits if it does have case. ASCII ended up encoding English into 7 bits and EBCDIC chose 8 bits (as it's based on a binary-coded decimal scheme which packs a decimal digit into 4 bits). Early machines did choose to use the unused high bit of an ASCII character stored in 8 bits as a parity bit, but most machines have instead opted to extend the character repertoire in a variety of incompatible ways, which eventually led to Unicode.
I wouldn’t be surprised if other machines had something like this in hardware.
Only if you assume a 1:1 mapping. But e.g. the original Baudot code was 5-bit, with codes reserved to switch between letters and "everything else". When ASCII was designed, some people wanted to keep the same arrangement.
[1] Yes, I remember you could bit-bang a UART in software, but still the parity bit didn't escape the serial decoding routine.
When your RAM is vacuum tubes or magnetic core memory, you don't want 25% of it to go unused, just to round your word size up a power of two.
wasnt this more to do with cost? they could do arbitrary precision code even back then. its not like they were only calculating numbers less than 65537, ignoring anything larger
Doing numbers little-endian does make more sense. It's weird that we switch to RTL when doing arithmetic. Amusingly the Wikipedia page for Hindu-Arabic numeral system claims that their RTL scripts switch to LTR for numbers. Nope... the inventors of our numeral system used little-endian and we forgot to reverse it for our LTR scripts...
Edit: I had to pull out Knuth here (vol. 2). So apparently the original Hindu scripts were LTR, like Latin, and Arabic is RTL. According to Knuth the earliest known Hindu manuscripts have the numbers "backwards", meaning most significant digit at the right, but soon switched to most significant at the left. So I read that as starting in little-endian but switching to big-endian.
These were later translated to Arabic (RTL), but the order of writing numbers remained the same, so became little-endian ("backwards").
Later still the numerals were introduced into Latin but, again, the order remained the same, so becoming big-endian again.
And as for numbers, perhaps it isn't too late to set it right once and for all. The French did that with the SI system after all.
> So apparently the original Hindu scripts were LTR
I can confirm. All Indian scripts are LTR (Though there are quite a few of them. I'm not aware of any exceptions). All of them seem to have evolved from an ancient and now extinct script named Brahmi. That one was LTR. It's unlikely to have switched direction any time during subsequent evolution into modern scripts.
But why? The brilliance of 8601/3339 is that string sorting is also correct datetime sorting.
To get the little-endian ordering. The place values of digits increase from left to right - in the same direction as how we write literature (assuming LTR scripts), allowing us to do arithmetic operations (addition, multiplication, etc) in the same direction.
> The brilliance of 8601/3339 is that string sorting is also correct datetime sorting.
I hadn't thought about that. But it does reveal something interesting. In literature, we assign the highest significance to the left-most (first) letter - in the direction opposite to how we write. This needs a bit more contemplation.
Yes, we do that with everything, which is why little-endian numbers would be really inconsistent for humans.
YYYY-MM-DD to me always feels like a timestamp, while when I want to write a date, I think of a name, (for me DD. MM. YYYY).
For better or worse, PowerPC is still quite entrenched in the industrial embedded space.
Hey, you! You're supposed to be dead!
May be someone can develop such thunking for legacy Linux userland.
that said, i sometimes think about a clean-room reimplementation of e.g. the unity3d runtime -- there are so many games that don't even use native code logic (which still could be supported with binary translation via e.g. unicorn) and are really just mono bytecode but still can't be run on platforms for which their authors didn't think to build them (or which were not supported by the unity runtime at the time of the game's release).
Yeah, that's a reasonable workaround, as long as it doesn't hit that OpenGL problem above (now it mostly affects DX7 era games, since they don't have Vulkan translation path). Hopefully it can be fixed.
The only thing I can think of is games, and the Windows binary most likely works better under Wine anyways.
There are many embedded systems like CNC controllers, advertisement displays, etc... that run those old applications, but I seriously doubt anyone would be willing to update the software in those things.
Win64S?
I think that there is a shitload of old desktop and laptop computers from 10 to 15 yrs that are still usable only with a linux distribution and that will not be true anymore.
Now Linux will be in the same lane as osx and windows running after the last shiny new things, and being like: if you want it, buy a new machine that will support it.
Lots of distros already dropped 32 bit kernel support and it didn't cause much fuss.
It's not even like we're breaking them. This is just the maintainers of the Linux kernel choosing to not spend their time maintaining compatibility with the old architectures.
A computer you buy today will be much more viable in 20+ years than a 20+ year old computer is right now. We were still in the extreme part of speed and density growth back than, and it will lessen every decade.
Houses are obviously very different to computers. Do you also demand 20 year lifetimes for your socks?
Maybe it is not that the architecture was not compatible as much as it was restricted or limited by Intel and co for these cpus
If you install 6.12 today (via e.g. Debian 13) then you'll be good until at least 2035. So removing it now de-facto means it will be removed in >10 years.
And as the article explains, this mostly concerns pretty old systems. Are people running the latest kernel on those? Most of the time probably not. This is really not "running after the last shiny thing". That's just nonsensical extreme black/white thinking.
From a user's perspective, they just keep working.
For mainstream laptops/desktops, the 32 bit era ended around 2006 (2003, if you were smart and using Athlon 64s instead of rancid Pentium 4).
Netbooks and other really weak devices held out a few years longer, but by 2010, almost everything new on the market, and a good chunk of the second-hand market, was already 64 bits.
Case in point, I'm writing on a x86_64 laptop that was a free give away to me about a year ago with a CPU release year that is 2012.
I have personally given away a x86_64 desktop unit years ago that was even older, might have had DDR1 memory.
Circa 2013 my old company was gifted a x86_64 motherboard with DDR2 memory that ended up serving as our in-office server for many years. We maxed the RAM (8GB) and at some point bought a CPU upgrade on ebay that gave us hardware virtualization extensions.
Same for WASM -- 32-bit pointers, 64-bit integers.
Both of these platforms have a 32-bit address space -- both for physical addresses and virtual addresses.
Ripping out support for 32-bit pointers seems like a bad idea.
RAM limitations were one reason to use arm64_32, but a bigger reason is that the first watches were only ARMv7 (32-bit) so by sticking with 32-bit pointers, Apple was able to statically recompile all the 3rd party (ARMv7) apps from LLVM bitcode to arm64_32.
https://www.macrumors.com/2025/06/16/watchos-26-moves-apple-...
It follows the latest LTS which I think is reasonable especially since phone vendors wants to have support for the device for several years.
The costs of distros and the kernel steadily dropping older x86 support over the last few years never causes an outcry but it's an erosion of what made Linux great. Especially for non-English speaking people in less developed countries.
Open-source maintenance is not a obligation, but it's sad there is not more people pushing to maintain support. Especially for the "universal operating system" Debian which was previously a gold standard in architecture support.
I maintain a relatively popular live Linux distro based on Ubuntu and due to user demand will look into a NetBSD variant to continue support (as suggested in this thread), potentially to support legacy 586 and 686 too.
Though a Debian 13 "Trixie" variant with a custom compiled 686 kernel will be much easier than switching to NetBSD, it appears like NetBSD has more commitment to longer-term arch support.
It would be wonderful to develop systems (eg emulation) to make it practical to support architectures as close to indefinitely as possible.
It does feel like a big end of an era moment for Linux and distros here, with the project following the kind of decision making of big tech companies rather than the ideals of computer enthusiasts.
Right now these deprecation decisions will directly make me spend time working at layers of abstraction I wasn't intending to in order to mitigate the upstream deprecations of the kernels and distros. The reason I have used the kernel and distros like Debian has been to offload that work to the specialist maintainers of the open-source community.
You can be so flexible that it turns everything into a giant slog. I doubt 32 bit support is anywhere near that horrid; commenting on the idea more than this specific case.
E.g. there's some stuff like erratum workarounds for old x86 CPUs that would be nice to drop, but they are part of a big framework for handling x86 diversity. Dropping individual workarounds doesn't let you drop the framework.
Exceptions are gonna be cases where dropping the support removed something significant from the lowest common denominator. Big stuff like word size, memory ordering (I assume dropping Alpha would be quite handy), virtual address space limitations.
Really the ALU width is an internal implementation detail/optimisation, you can tune it to the size you want at the cost of more cycles to actually complete the full width.
Lots of machines are capable of running with 32-bit pointers and 64-bit integers ("Knuth mode" aka "ILP32"). You get a huge improvement in memory density as long as no single process needs more than 4GB of core.
But really that's a software/OS level thing, and though the benefits have definitely been shown, the seem small enough to not be worth the upheaval.
Though possibly related, larger pages have been shown to have significant speedups without changing the ABI (as much, at least mmap() and similar will have slightly different specifics). IMHO the only possible "benefit" to 4kb page sizes is to (ab)use it to trap things like array overruns - though using that is a poor substitute for /real/ bounds checking - a lot can go wrong within 4kb, after all.
Part of the problem with these discussion is that often when people say "64-bit" vs "32-bit" they are also considering all the new useful instructions that were added to the new instruction set generation. But a true "apples-to-apples" comparison between "32-bit" and "64-bit" should be using almost identical whose only difference is the datapath and pointer size.
I feel that the programs and games I run shouldn't really need more than 4GB memory anyway, and the occasion instance that the extra precision of 64-bit math is useful could be handled by emulating the 64-bit math with the compiler adding a couple extra 32-bit instructions.
That's what the Raspberry Pi Desktop did.
It's the Pi project's x86 OS, a drastically cut-down Debian with LXDE. It was pretty much the smallest full-desktop distro I've seen.
(Excluding specialist things with window managers like antiX or TinyCore.)
Sadly never made it off Bookworm. I wish they'd update it, or someone took over maintenance.
When you factor in memory fragmentation, you really only had a solid 0.75-1.5GB of space that could be kept continuously in use. That was starting to become a problem even when 32-bit was the only practical option. A lot of games saw a benefit to just having the larger address space, such that they ran better in 64-bit with only 4GB of RAM despite the fatter 64-bit pointers.
I think it would be possible for e.g. microkernels to greatly reduce the size of the reservation (though not to eliminate it entirely). However, I can't imagine how you would handle the privilege escalation issue without having at least some system code in the application's virtual address space that's not modifiable by the application.
Such a design may require at least one processor dedicated to running the kernel at all times, so it might not work on a single processor architecture. However, single processor architectures might be supportable by having the "kernel process" go to sleep by arming a timer and the timer interrupt is the only one that's specially mapped so it can modify the page table to resume the kernel (for handling all the ring buffers + scheduling). As you note, there's some reserved address space but it's a trivial amount just to be able to resume running the kernel. I don't think it has anything to do with monolithic vs microkernels.
If you just want constant-time operations, you just need dedicated instructions. Maybe they run on dedicated cores, but that's an implementation detail. I think this is a good idea, and it feels like we're heading this way already, more or less.
If you want to address all of the vulnerabilities that have arisen, you need full isolation. As mentioned in a sibling comment, you can't share any memory or caches etc. Each "security core" would have to be fully isolated from every other core, including every other security core. And you also need to segregate out "sensitive" code and data from non-sensitive code and data. When all is said and done, I don't see how we're talking about anything less than scrapping all existing ISAs, and perhaps even von Neumann architecture itself. I'm sure there are some middle-ground solutions between where we are today and this extreme, but I think they're going to look more like TPM/Secure Enclave than special cores, and they're only going to partially address the vulnerabilities.
And 3-1 wasn't really experimental. It was essentially always that way under Linux, and had been supported under Windows since the late 90s.
I couldn't tell if your comment was a joke, but it is worth mentioning the 8-bit microcontrollers like TinyAVR still fill a niche where every joule and cent counts.
Compared to ARM-based systems? Nope.
I saw this last in SmartOS.
I'm pretty shocked to see comments like "the RAM for a 32-bit system costs more than the CPU itself", but open source isn’t supposed to be about market pricing or what’s convenient for vendors; it’s about giving users the freedom to decide what’s worth running.
I understand that maintainers don’t want to drag around unmaintained code forever, and that testing on rare hardware is difficult. But if the code already exists and is working, is it really that costly to just not break it? The kernel's history is full of examples where obscure architectures and configs were kept alive for decades with minimal intervention. Removing them feels like a philosophical shift, especially when modern hardware is more locked down and has a variety of black box systems running behind it like Intel ME and AMD PSP.
Ehhh, it's about users having the ability to run whatever they like. Which they do.
If a group of users of 32 bit hardware care to volunteer to support the latest kernel features, then there's no problem.
If no one does, then why should a volunteer care enough to do it for them? It's not like the old kernel versions will stop working. Forcing volunteers to work on something they don't want to do is just a bad way to manage volunteers.
The other dynamic here is that the direction in Linux does come from the top. When you have maintainers like Arnd Bergmann saying they would "like" to remove support for hardware (like the ARM boards), that sets the tone, and other contributors will naturally follow that lead. If leadership encouraged a philosophy closer to "never break existing hardware" the same way we’ve had "never break userspace" for decades, we probably wouldn’t even be debating removing 32 bit.
I’m not saying kernel devs need to carry the weight alone, but it would be nice if the community’s baseline stance was towards preservation rather than obsolescence. :(
It's not just the case that you need people to support 32bit/nommu; you also have to account for the impact on other kernel devs working on features that are made harder.
This is called out in the article around keeping highmem support.
There's not a compelling reason to run a bleeding edge kernel on a 2004 computer, and definitely not one worth justifying making the kernel devs support that setup.
But in any case, I’m sure Red Hat etc would be happy to sell backports of relevant fixes.
It’s not that I’m unsympathetic to people with older systems. I get it. I’ve got old hardware floating around that I’ve successfully kept my wife from ecycling. It’s that I’m also sympathetic to the kernel devs who only have so many hours, and don’t want to use them supporting ancient systems that aren’t still widely used.
For legacy hardware like this it's usually not anywhere close to as important as it is for modern systems.
These systems are not being used to browse the modern web, they're too slow.
They're not being used to host production multiuser environments beyond a few retrocomputing homelabbers' toy systems, where the worst case scenario is a restore from backup and banning a user or two rather than data loss/breach and/or legal action.
The ones still in actual active use are almost all appliance systems that mostly haven't seen a kernel update in years or ever because they are usually exist to go with some piece of hardware that never got an in-tree driver and thus can't work with anything much newer than what it shipped with and/or some software that depends on ancient libraries no distro ships anymore. These systems don't need to (and shouldn't) be exposed to untrusted networks, users, or content, they can (and already should) be locked down by a skilled admin to only communicate with the minimum number of systems needed for whatever purpose they serve. If the admin isn't sufficiently skilled to confidently handle that, the system and admin should both be replaced.
---
I have an old IBM PS/2 that's my family's first computer, which still has its original Windows 3.1 install on it. I imaged the original hard drive and moved it to a CF card, but that also means I can screw around with it and not worry about breaking anything because I can just restore the last known good image. I don't connect it to the internet often but if on one of those rare times I happened to somehow stumble upon someone who had been saving a drive-by exploit for IE 3.0 or a RCE against Trumpet Winsock that then infected my system I'd just do the same. Anything this old is small enough to be imaged easily.
It depends on the feature, but in many cases the answer is in fact 'yes.' There's a reason why Alpha support (defunct for decades) still goes on but Itanium support (defunct for years) has thoroughly been ripped out of systems.
Well, there really wasn't much support for atomic instructions in x86 before the introduction of compare-exchange in the 486. Any time you wanted guaranteed atomicity on a 386 you had to disable interrupts, which among other things means that if you lock up during a critical section your entire system hangs. Another implication is that nearly all of our lockless data structure constructs depend on compare-exchange instructions.
It vastly simplified some very tricky sections of the kernel to remove support for systems that don't have hardware atomic instructions, so it ended up being done.
Not really. The discussion is about cost, benefits and available resources. Projects are not immune because they are open source or free software. Actual people still need to do the work.
> Open source has always been about making hardware outlive commercial interest and let it run long after the hardware vendor abandons it.
Again, not really. Open source has always been all about freely modifying and distributing software. This leaves some freedom for anyone to keep supporting their pet hardware, but that’s a consequence. In this case, I don’t think it would be a real problem if anyone would step up and commit the ressources necessary to keep supporting older hardware. This freedom was not taken away because a project’s developers decided that something was not worth their time anymore.
> The kernel is still adding support for some 32-bit boards, he said, but at least ten new 64-bit boards gain support for each 32-bit one.
And
> To summarize, he said, the kernel will have to retain support for armv7 systems for at least another ten years. Boards are still being produced with these CPUs, so even ten years may be optimistic for removal. Everything else, he said, will probably fade away sooner than that.
So, no, he does not think that at all.
Wild, like some kind of virtual cache. Reminds me a bit of the old Macintosh 68k accelerators; sometimes they included their own (faster) memory and you could use the existing sticks as a RAM disk.
$ cat /proc/version
Linux version 2.6.18-419.0.0.0.2.el5PAE ... (gcc version 4.1.2 20080704 (Red Hat 4.1.2-55)) #1 SMP Wed Jun 28 20:25:21 PDT 2017The "steam client" is still a 32 bits ELF executable, which statically loads openGL and x11 libs... (namely not even a wayland->x11 fallback or a opengl->CPU rendering).
We would be all better with a nogfx static PIE executable, even a nogfx dynamic PIE executable if they want to explore the ELF setup of a distro.
Ohhh yes!
So, a couple of weeks ago I came across a discussion where some distro (I don't remember which one) contemplated removing 32-bit user space support, suggesting to users to simply run a VM running a 32 bit Linux instead. It was a stupid suggestion then, and this statement is a nice authorative answer from the kernel side, where such suggestions can be shoved to.
I've been pushing hard for us to move off SLES as a result, and I do not recommend it to anyone who wants a stable distribution that doesn't fuck over its users for stupid reasons.
"Linking against 32-bit multilib packages has been removed. The *.i686 packages remain supported for the life cycle of Red Hat Enterprise Linux 9."
https://docs.redhat.com/en/documentation/red_hat_enterprise_...
Like on Armv7-M it's said "Nobody is building anything with this kind of hardware now" - this is just wrong to the point of ridiculousness. Thousands of new products will be designed using these microcontrollers and still billions of units will be produced with them in them - now, true that almost none of those will run Linux on those MCUs but it's crazy to say "nobody" is building things with them. Many of course are moving to Armv8-M microcontrollers but those are 32 bit too!
On the Linux side, there are things like the AMD/Xillinx Zynq-7000 series that will be supported for many years to come.
It's not the worst idea in the world to deprecate support for 32-bit x86 but it is not time to remove it for ARM for many years yet.
2. That sentence wasn't about 32-bit, it was about devices without MMUs.
I am also surprised how little attention the 2038 problem gets. However, I also wonder how big a problem it is. We've known about it for years and all the software I've touched is not susceptible to it.
I'm sure it'll cause someone some issues, but it'll be very niche by 2038. Most of the sorts of systems that it could cause problems for aren't being used to plan things years in advance, some probably don't even do anything with the date. So it's a niche within a niche (unupdatable systems used for future planning) that's likely to have problems soon, and overall probably no longer a big deal.
TL;DR: It's solved upstream, only an issue for some systems that don't get updates.
date -u --date='2038-01-19T03:14:08' +%s | perl -ne 'printf "%#x\n", $_'
[1]: https://perldoc.perl.org/Time::Piece#Use-of-epoch-seconds