I've actually been pondering this problem for a couple of years - when everything is based on FLASH, what do you do when something corrupts your "base install"?
The VIC-20, C64, Timex-Sinclair and other computers of their day had the OS in ROM that couldn't be changed. So you couldn't break these computers via the keyboard (or other peripherals) but if there was a bug somewhere in the firmware (which included the OS and BASIC) you had to live with it for the life of the device.
You can get back to those days via emulation - https://vic-20.appspot.com/emus.htm, but there's nothing (that I know of) that's a physical device with these characteristics.