Assembly requires way more work than compiling to, say C. Clang and gcc do a lot of the heavy lifting regarding optimisation, spilling values to the stack, etc
I mean, I could have used the C stack as the VM's stack but then you have to worry about blowing up the stack, not having access (without a bunch of hackery, looking at you scheme people) to the values on the stack for GC and whatnot and, I imagine, all the other things you have issues with but it's not needed at all, just make your own (or, you know, tail call) and pretend the C one doesn't exist.
And I've started on another VM which does the traditional stack thing but it's constrained (by the spec) to have a maximum stack depth so isn't too much trouble.