Old computers contained firmware by the way - BASIC and the OS weren't loaded from disk, they were stored on internal ROM.
The only reason built-in functionality didn't change, was again just due to the lack of WAN connectivity. You can achieve the same level of privacy today by just not connecting to the internet and using your machine offline only.
That said, this is something I did with a cheap laptop for a while—installed a basic Linux and Emacs and used it as a mostly-isolated Emacsing machine; I believe I never hooked it up to the network again, though I did sketch out some provisions for manually physically disconnecting sensitive filesystems in case I needed to do that very occasionally (while being aware that sufficiently clever malware could still work around this). But then the logic board malfunctioned much sooner than I'd expected, and I haven't been in a position to get another since.
And 100% air-gapping is not always desirable. Many 1990s-and-before machines that had networking capabilities were still the epitome of “local first” by comparison to the “deployment is primarily done through progressive Web apps” world of 2021. So you could have a few channels by which to talk to the outside world, but without the outside world getting its hooks deep inside you in the process: the television without the telescreen. (Security issues aside, of course—though some of that is mitigable with restrictive proxying, simpler protocols, etc. depending on how retro and how networked you are trying for.)