Why the 6502 instead of the 65816?
The segmented architecture is one thing, but the painful way it was hacked into the instruction set is not at all enjoyable to code for.
a) You're constantly having to flip modes between 16 and 8 bit for various registers, and keep track of which mode you're in.
b) There's no register that can hold a full 24 bit address. So imagine doing bitmap graphics in, say, a 640x480 display when you can only directly address the first 64k of it before having to futz with segments.
c) The thing sucks to interface. Many people have solved this problem but having to demultiplex the bus is not fun, especially when the 65xx already does so much in a single cycle.
The 816 was a hack, but not a particularly brilliant one. It does add some nice things (relocatable base page, etc.), but the 6502's instruction set didn't really have room for doing it right. So they didn't.
Have you ever looked at the 65C832? It never made it to production but there are detailed specs and a datasheet; probably enough for someone to implement one on FPGA or something. I'm curious if it would have fixed some of the issues you report.
https://downloads.reactivemicro.com/Electronics/CPU/WDC%2065...
But it still has the problem of futzing around between modes for the registers, modal instruction set... terrifying. And the program counter is still inexplicably 16-bits, so have fun writing a compiler for programs bigger than 64k, you have to manually manage the bank register when you move between banks.
And looks like direct page and stack still stuck in the bottom bank? At least I think I read that?
And for god's sake, why multiplex the databus still, like the 816?! Why wouldn't they just make a 64-pin DIP version like Motorola did with the 68000?
Anyways, it never made it to production. But basically looks like an 816 x 2, which isn't terrible in that having double the size on those registers makes doing a lot of things a lot easier.
With sharing I mean, when you have two or more tasks of which only one instance will run, they can use the same direct page. Similar if you can statically limit the stack size of the tasks to a fraction of 256 bytes, you can fit multiple stacks on the same page with some work.
Program counter is 16 bit, with a separate bank byte; it'll wrap when it hits the bank boundary.
Constantly having to flip between register modes is a hassle. And makes it very difficult to write compilers and high level languages. Many common patterns require flipping back and forth a few times.
Either they needed to have a whole new set of addressing modes, or they should have just added 3 new registers -- 16 bit versions of X Y and A -- and let the two co-exist.
Honestly, this is an era when a 68000 could be had with a full 16 register set of 32-bit registers. The 65816 doesn't look so hot in that light, though it certainly was faster at interrupt handling.
I guess Nintendo made do with their modified 816, but it really is not a nice architecture.