Hacker News Reader

Comment by narag | Hacker News Reader

This brought memories of that pattern (flyweight?) where the data was stored outside the objects, possibly in an array. An object was instantiated only to hold an index to the array position and allow access. That's dirty cheap!

View on HN

It's still commonly used. The NFX library has Pile which does this well for holding large data:

http://nfxlib.com/book/caching/pile.html

https://www.infoq.com/articles/Big-Memory-Part-1

This pattern was also used by Java and .NET for implementing cheap String.substring calls where all substrings would use the same underlying array with just offsets changed. Unfortunately it turns out that people read entire files into a one big String and then have a reference to just a small piece of it (via substring) marking the big underlying array as reachable for the GC holding a lot of memory for no reason. That's why new implementations of substring always copy:)

I know this was changed recently-ish in Java, but I hadn't heard of anybody doing the old substring trick in .NET, do you know when they cut over?

There is a lot of good things going on in .NET Core about this. Here is a good description of Span: http://adamsitnik.com/Span/#slicing-without-managed-heap-all...

AFAIK this trick was never possible in .NET because Substring always 'deep copied' the relevant data into a completely new string

Regex's implementation in the standard library used to do this, and I think maybe still does. Bonus points because they leaked a reference to the last string you ran a regex against (????) so if it was big you'd just eat up 50 mb of heap for a while.

That pattern can be done in .NET via native memory allocation (MarshalInterop and SafeHandles).

With the latest C# 7.x features will become easier to use it.

QPX also stored their database in non-GC space in order to prevent the GC from having to walk it.

It's a common pattern — e.g. a lot of C++ libraries have some sort of string piece type.

good old fashion separating data from logic.

Reply on news.ycombinator.com