rptb1

It would be excellent to have the Definition available online. MLWorks originated with a contract to develop a very strict implementation of the Definition, and we were quite proud of being truly Standard ML.

Why thank you. Although we've opened the coffin, we've yet to reanimate the corpse. Anyone who'd like to help out, please take a look at https://github.com/Ravenbrook/mlworks/wiki/Roadmap

I think we'd need a more careful definition of "handle" there!

And we're definitely not proselytizing garbage collection here. The MPS is a framework for both manual and automatic memory management (and co-operation between the two). One of our main high performance commercial applications is all about the manual management, and for very good reasons.

But nobody should be rejecting GC out of hand, that's for sure.

This is a really interesting topic and I could fill your screens with a wall of text. It's true that the MPS currently does not collect concurrently, however the only thing that makes it not-concurrent is a critical point in the Shield abstraction where the MPS seeks to gain privileged access to memory (usually in order to scan it for GC). The critical point is where ShieldExpose in shield.c has to call ShieldSuspend to preserve the shield invariants. This is the only point in the MPS that prevents concurrency, and the rest of the MPS is designed to support it.

The restriction could be removed if either:

  * the MPS could use a different set of protections to the mutator program

  * the mutator program uses a software barrier

The first one is tricky, and the second one just hasn't come up in any implementation we've been asked to make yet. Given a VM, it could happen, and the MPS would be concurrent.

So, I believe there's nothing fundamentally non-concurrent about the MPS design. It's kind of waiting to happen.

[I've put this in a comment in the code. Thanks for making me write it down :)]

There is currently a "global pause" to scan thread registers and stacks at "flip", but that's all. Is that what you meant?

Since we're just moving our commercial clients onto 64-bit, we don't have experience except up to the 4GB limit (or 3GB in old Windows) which they approach regularly. We're still waiting for it to become a problem. I may have a different opinion in a year.

One of my short term goals is just to get some measurements together to publish. Watch this space, and sorry that they aren't here yet.

Perhaps you could write to mps-questions and we can have a chat about it.

Sure. The MPS approach that's deployed commercially is to use an unfashionable hardware read barrier to amortize the cost of the pointer rewrite, allowing the heap to be compacted incrementally. There's nothing that takes minutes or even seconds -- the cost is spread through the entire execution. In some sense, all this can take "minutes" of real time, but not at the cost of stopping the program doing stuff. The MPS design has always been to spread the cost evenly, not push it into a cataclysmic event in the future.

That said, we have an abstract framework that could be made to work in other ways, depending on requirements. It wouldn't be much work to hook in a write-barrier-only pool class, etc. etc. and have it co-operate. However, most of the development effort so far has gone on the read barrier approach.

As to how this effects overall run-time, well, that's where we'd have to arrange a side-by-side comparison, and make sure it included one of those compactions :P

Yes indeed I'm very aware of this. The overall architecture and abstractions can support efficient multi-core operation, but the implementation is behind. Development is mostly paid for by single-threaded clients at the moment! But I have plans. Well spotted, by the way.

Most of the problems are around the bottleneck of stopping threads on other cores in order to get privileged access to memory while preserving their consistent view of the heap. What the MPS needs is its own protection map. Most OSs don't help much with that, although Mac OS X, for a while, allowed you to map the same physical RAM at two addresses with different protections. Sadly, no longer.

Old but still reasonably accurate for a high-level view is the original "open source" announcement paper http://www.ravenbrook.com/project/mps/doc/2002-01-30/ismm200...

I can't give you figures right now, but the reason we have commercial clients for the MPS is that we have extremely low pause times. Side-by-side comparisons are expensive to arrange. I'm working on that :)

Yeah we're not really a drop-in malloc replacement like Boehm, though we could have some glue which deploys the MPS like that. Take a look at the Scheme example to see how the MPS integrates at present http://www.ravenbrook.com/project/mps/version/1.110/example/...

It's both a precise and conservative GC, depending on what you declare to it, and which pool classes you use. In commercial deployment and in Open Dylan it's a mostly-copying GC (i.e. a mixture of both) see http://www.memorymanagement.org/glossary/m.html#mostly-copyi...

I should probably clarify that little bit of hyperbole. We do track known issues (of course), but bugs in production are extremely rare -- about one per year.

It's an extremely simple toy intended to help people understand how to integrate the MPS, rather than show it off. But have at it!