Comment by whartung | Hacker News Reader

Comment by whartung | Hacker News Reader

Parent Full thread

Ive always "dreamed" of having something like the ST image but backed by persistent virtual memory, not simply RAM.

(Crudely) mmap the ST image to a 100GB file and just change pages. Let the OS flush the pages back and forth.

Maybe full boat, heap sweeping GCs would be Bad as it pages the entirety of the heap in and out. But we have (had) lots of RAM these days. We have generational GCs that leave idle stuff alone. Its no doubt impractical, but I think it would be neat to have the entirety of my historical email in the global "mailbox" array, and I can build indexes off it as I wished. But the mail isn't "on disk", it's all marshalled up as first class objects.

I don't know enough about it, no doubt it "won't work", but the idea of simply mapping the entire, large, ST VM heap to a persistent backing store, just be interesting I would think.

Dangerous too, as there is a demarcation between "running image" and "saved image". But, still think it could be interesting.

Hehe, this was kind of my thesis back 2015; non public unfortunately, but I think the following is save to share: So basically they had the need to persist huge collections of objects that would not fit into a single servers RAM. As for persistence, a key value NoSQL db was chosen (which is a natural fit for a world where everything has a unique identity - I also have some dislike for ORMs because of that, because I saw how beautiful it could synergize, and how ugly some mechanism are that hibernate, EC Framework etc. have to implement in order to brigde the gap between objects and relational dbs).

The core pattern was that the Proxy, which was under Nil and above all other Objects regarding inheritance I think.

It re-implemented `doesNotUnterstand:` by: 1) First loading the actual object from the persistent storage and 2) sending the not understood message to the actual loaded object (which might be able to answer the message instead of calling `doesNotUnderstand:` for every message, like Proxy did.

There where if course optimisations and so on, but that was the gist of it.

What I really like about this system was that it was completely transparent to the sender of the message whether they were talking to a proxy, or the already-loaded object, all while being robust, easy to maintain and so ob. Dealing with collections was tricky though (how much to load at once? what about searching for a particular object?...), and would have been aswell for deeply nested object (which they successfully avoided though because as a SaaS-company, they could model the data exactly to their needs).

I'm also interested in this and made decent progress on a git-style content addressed image where everything is identified by hash and residency determines what's in memory. For example, there's a filesystem on the image that works this way and you can page over the "cold" non-resident bytes for very large files that you would not want to hold in memory.

You don't persist the state of the full image continuously to the host disk, but any given state (new programs, new runtime state, filesystem) can be saved via a delta, not a full rewrite, due to this representation.

So this is a little different than what you are talking about, but I'd say it's possible.

Sounds like you would find NixOS and/or Unison interesting!

Yes, big fan of Unison (the system I described above follows its model for code hash identity) and I use a NixOS machine for most of my software work!

Sounds a somewhat like GemStone/S

I think also IBM has or had somewhat similar concepts.

https://en.wikipedia.org/wiki/GemStone/S

Gemstone/S was first an object database, per above article. Vaguely reminds me of:

https://en.wikipedia.org/wiki/InterSystems_Cach%C3%A9

Which I heard about from a friend who was using it at work.

https://en.wikipedia.org/wiki/Comparison_of_object_database_...

Caché is first one in the table.

I remember object or object relational databases were popular at one time. maybe they still are to some extent.

Did you dream of ColdStore (https://coldstore.sourceforge.net/rationale.html) ?

Reply on news.ycombinator.com