1. Using a simple prototype VM seemed easier during initial development
2. When the language was developed, no suitable common VM existed
3. The language developers thought they could do it better for their specific case, i.e. the point of the implementation is to showcase new VM strategies
Remember that a language and an interpreter are separate things, and usually an author who wants to try an idea for one has to find some suitable stand-in for the other in order to get started.
It looks like we're at a point now where many dynamic languages are attempting to transition to a more advanced VM. This includes Perl, Python, Ruby, and some implementations of JavaScript. I assume they all want to take advantage of the latest VM techniques, and they're all making major changes anyway, so again I just wonder why they don't seriously consider collaborating now.
The Jython project was just the coolest darn thing until, from what I can tell, the lead developer got tired of maintaining a mature project with a hairy codebase and left to join (start?) PyPy, an even more ambitious project. So Jython was stuck in a coma with a pre-alpha 2.2 release for like half a decade, while CPython marched on, and no new maintainer was able to bring it back to consciousness -- despite huge demand for it, since Jython at the time was the perfect solution to problems in Java and Python. Finally Frank Wierzbicki came along a couple years ago, squeezed out the stubborn 2.2 release and is on the verge of version 2.5, at which point we can consider Jython back open for business. There's a kind of similar story with Psyco, and I think that one also concludes with PyPy being the "right answer" to the original problem.
But back on topic -- in some ways, the dynamic-language communities are collaborating on common VMs. The JVM team is trying to make things easier for other languages with "invokedynamic", the .NET world is keeping up at least (DLR? dunno), and LLVM has good traction as a compilation target.
However, I doubt we'll see anyone devote significant effort to porting Python or Ruby to Parrot until Perl 6 lands and provides a compelling demonstration. Meanwhile, PyPy is getting close enough to its goal that we might just want to wait and see if their JIT-compiler-generator solves all of these problems in one shot.
Original comment: Yeah, that would be one solution, but I think the shared VM helps so you can easily access libraries written in different languages. I remember why (of Ruby fame) trying to run Ruby code on Google App Engine back when it was Python-only. He discovered that Ruby bytecode is nearly identical to Python bytecode. Like me, he didn't understand why Python and Ruby still need to be on separate runtimes.
(That was sarcasm.)
Note that the JVM and .NET runtimes do pretty much exactly what was asked--they run programs written in multiple languages on the same VM. Because the "native" VMs for Python and Ruby are so bad, the JVM and .NET VMs can even out-perform the native ones for most tasks.
The hard part comes when you want to preserve source-level semantics and provide a natural-feeling translation so that a human can understand, read, and write the source.
However, with a VM you don't have that problem. With a VM, a lossy, one-way transform is sufficient. You can easily throw away information as you compile, since you're not targeting humans consumption for the output. You can replace, say, overloaded functions with mangled names. You can turn multiple dispatch into decision trees of conditional statements. And so on. With a VM, you would want to keep enough high-level detail to efficiently JIT stuff, and it's a bit of a challenge to decide what detail is sufficiently useful to include, but it's not insurmountable.
That's what I meant. As far as I know, nobody else is taking it seriously as a primary VM for their language.
Is Parrot 1.0.0 radically better than other (non-perl) VMs?
It's too low level to do compile to directly. It'd occupy a place in the stack somewhere near assembly, but with an optimizer in place. It's more of a VM toolkit than a VM itself.