Past me, however, did things like write an OTP generator for TI-83+ partly on the grounds that it was a non-networked device and simple enough to leave less in the way of interesting ways for malware to hide.
I’m most familiar with old Mac OS machines. Like, before it was called Mac OS. And they are extremely difficult to connect to anything on the contemporary internet, because none of the available browsers have the ability to work with contemporary security certificates.
George RR Martin still writes novels on a DOS machine with WordStar 4.
There are some other holdouts but I imagine for any not-bestselling author there's an intermediate step on a modern machine to convert files to a more modern format.
Old computers contained firmware by the way - BASIC and the OS weren't loaded from disk, they were stored on internal ROM.
The only reason built-in functionality didn't change, was again just due to the lack of WAN connectivity. You can achieve the same level of privacy today by just not connecting to the internet and using your machine offline only.
That said, this is something I did with a cheap laptop for a while—installed a basic Linux and Emacs and used it as a mostly-isolated Emacsing machine; I believe I never hooked it up to the network again, though I did sketch out some provisions for manually physically disconnecting sensitive filesystems in case I needed to do that very occasionally (while being aware that sufficiently clever malware could still work around this). But then the logic board malfunctioned much sooner than I'd expected, and I haven't been in a position to get another since.
And 100% air-gapping is not always desirable. Many 1990s-and-before machines that had networking capabilities were still the epitome of “local first” by comparison to the “deployment is primarily done through progressive Web apps” world of 2021. So you could have a few channels by which to talk to the outside world, but without the outside world getting its hooks deep inside you in the process: the television without the telescreen. (Security issues aside, of course—though some of that is mitigable with restrictive proxying, simpler protocols, etc. depending on how retro and how networked you are trying for.)